[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:03 dist-test-slave-p39x krb5kdc[11905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:03 dist-test-slave-p39x krb5kdc[11905](info): set up 2 sockets
Sep 26 16:37:03 dist-test-slave-p39x krb5kdc[11905](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 16:37:03.998795 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:04.143874 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:46703
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:33493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:46703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:04.253434 11918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:04.253769 11918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:04.253854 11918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:04.257966 11918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:04.258074 11918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:04.258111 11918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:04.258149 11918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:04.258205 11918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:04.263784 11918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:33493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:46703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:46703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:04.265048 11918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:04.266114 11918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:04.275282 11926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:04.275290 11923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:04.275295 11924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:04.275342 11918 server_base.cc:1034] running on GCE node
I20260926 16:37:04.276118 11918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:04.277323 11918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:04.278484 11918 hybrid_clock.cc:648] HybridClock initialized: now 1790440624278467 us; error 31 us; skew 500 ppm
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:04.281329 11918 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:04.282280 11918 webserver.cc:460] Webserver started at http://127.11.158.190:43383/ using document root <none> and password file <none>
I20260926 16:37:04.282820 11918 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:04.282881 11918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:04.283042 11918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:04.285126 11918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "bf86b326695143478cf5cf9231d9dcca"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "cdf03e2943200e0f66f1c538ccfeba5e"
server_key_iv: "ef0312d4c1a44ee7d7f0d602d9672a3a"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.285624 11918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "bf86b326695143478cf5cf9231d9dcca"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "cdf03e2943200e0f66f1c538ccfeba5e"
server_key_iv: "ef0312d4c1a44ee7d7f0d602d9672a3a"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.289438 11918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:37:04.291884 11933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:04.293210 11918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:04.293313 11918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "bf86b326695143478cf5cf9231d9dcca"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "cdf03e2943200e0f66f1c538ccfeba5e"
server_key_iv: "ef0312d4c1a44ee7d7f0d602d9672a3a"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.293459 11918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:04.306445 11918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:04.309198 11918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:04.309386 11918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:04.316232 11918 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:46703
I20260926 16:37:04.316259 11985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:46703 every 8 connection(s)
I20260926 16:37:04.317186 11918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:04.319932 11986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:04.320251 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 11918
I20260926 16:37:04.320425 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:04.321691 11898 external_mini_cluster.cc:1428] Setting key e7da1403690a24254cdbef12e6d49074
I20260926 16:37:04.326157 11986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca: Bootstrap starting.
I20260926 16:37:04.329017 11986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:04.329955 11986 log.cc:824] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:04.332299 11986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca: No bootstrap required, opened a new log
I20260926 16:37:04.336762 11986 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf86b326695143478cf5cf9231d9dcca" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46703 } }
I20260926 16:37:04.337064 11986 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:04.337116 11986 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf86b326695143478cf5cf9231d9dcca, State: Initialized, Role: FOLLOWER
I20260926 16:37:04.337585 11986 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf86b326695143478cf5cf9231d9dcca" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46703 } }
I20260926 16:37:04.337720 11986 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:04.337812 11986 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:04.337929 11986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:04.339212 11986 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf86b326695143478cf5cf9231d9dcca" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46703 } }
I20260926 16:37:04.339587 11986 leader_election.cc:302] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf86b326695143478cf5cf9231d9dcca; no voters:
I20260926 16:37:04.339972 11986 leader_election.cc:288] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:04.340169 11991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:04.340488 11991 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [term 1 LEADER]: Becoming Leader. State: Replica: bf86b326695143478cf5cf9231d9dcca, State: Running, Role: LEADER
I20260926 16:37:04.340870 11991 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf86b326695143478cf5cf9231d9dcca" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46703 } }
I20260926 16:37:04.341432 11986 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:04.342613 11993 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf86b326695143478cf5cf9231d9dcca. Latest consensus state: current_term: 1 leader_uuid: "bf86b326695143478cf5cf9231d9dcca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf86b326695143478cf5cf9231d9dcca" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46703 } } }
I20260926 16:37:04.342744 11993 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:04.342614 11992 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf86b326695143478cf5cf9231d9dcca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf86b326695143478cf5cf9231d9dcca" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46703 } } }
I20260926 16:37:04.343081 11992 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:04.343156 12000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:04.346817 12000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:04.349320 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:04.324787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.11.158.190:46703)
0926 16:37:04.325306 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:04.325325 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:04.326431 (+ 1106us) server_negotiation.cc:407] Connection header received
0926 16:37:04.330833 (+ 4402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:04.330861 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:04.331172 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:04.331454 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:04.334655 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.335623 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.337368 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.338002 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.338154 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:04.343318 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:04.343328 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:04.343352 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:04.343394 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:04.345727 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:04.346316 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:04.346322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:04.346328 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:04.346471 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:04.346826 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:04.346830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:04.346831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:04.347354 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:04.347562 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:04.348182 (+ 620us) server_negotiation.cc:299] Negotiation successful
0926 16:37:04.348471 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":157,"threads_started":1}
I20260926 16:37:04.352942 12000 catalog_manager.cc:1347] Generated new cluster ID: a9dc0fde1d084271a5e4f56719b2f74c
I20260926 16:37:04.353003 12000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:04.359962 12000 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:04.360967 12000 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:04.372668 12000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf86b326695143478cf5cf9231d9dcca: Generated new TSK 0
I20260926 16:37:04.373519 12000 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:04.412115 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46703
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:33493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:04.527638 12012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:04.527935 12012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:04.528017 12012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:04.532078 12012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:04.532168 12012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:04.532274 12012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:04.537443 12012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:33493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:04.538479 12012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:04.539515 12012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:04.548866 12017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:04.548969 12012 server_base.cc:1034] running on GCE node
W20260926 16:37:04.548902 12020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:04.548902 12018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:04.549603 12012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:04.550210 12012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:04.551397 12012 hybrid_clock.cc:648] HybridClock initialized: now 1790440624551376 us; error 43 us; skew 500 ppm
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:04.554068 12012 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:04.554893 12012 webserver.cc:460] Webserver started at http://127.11.158.129:36267/ using document root <none> and password file <none>
I20260926 16:37:04.555440 12012 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:04.555531 12012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:04.555755 12012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:04.557796 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "ae57ae341fe54302b6fef4d5e187f46f"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "ff88086d12bc4d79d434090657dda368"
server_key_iv: "53702b702a5e90cb8e9c0cfc57247203"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.558413 12012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "ae57ae341fe54302b6fef4d5e187f46f"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "ff88086d12bc4d79d434090657dda368"
server_key_iv: "53702b702a5e90cb8e9c0cfc57247203"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.562091 12012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:37:04.564538 12027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:04.565937 12012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:04.566068 12012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "ae57ae341fe54302b6fef4d5e187f46f"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "ff88086d12bc4d79d434090657dda368"
server_key_iv: "53702b702a5e90cb8e9c0cfc57247203"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.566185 12012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:04.578106 12012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:04.581403 12012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:04.581605 12012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:04.582193 12012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:04.583009 12012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:04.583078 12012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:04.583148 12012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:04.583197 12012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:04.591795 12012 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:38185
I20260926 16:37:04.591825 12140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:38185 every 8 connection(s)
I20260926 16:37:04.592801 12012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:04.599085 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12012
I20260926 16:37:04.599195 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:04.599460 11898 external_mini_cluster.cc:1428] Setting key d5a2224738966753fe1e232c7df78942
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:37:04.608410 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:04.594614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:54893 (local address 127.11.158.190:46703)
0926 16:37:04.594760 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:04.594764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:04.596662 (+ 1898us) server_negotiation.cc:407] Connection header received
0926 16:37:04.597588 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:04.597590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:04.597643 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:04.597773 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:04.599073 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.599734 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.601289 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.601521 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.601632 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:04.604677 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:04.604680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:04.604682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:04.604707 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:04.606231 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:04.606793 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:04.606797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:04.606798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:04.606872 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:04.607235 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:04.607239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:04.607240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:04.607429 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:04.607536 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:04.608081 (+ 545us) server_negotiation.cc:299] Negotiation successful
0926 16:37:04.608214 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:37:04.609301 12143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:04.595041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:46703 (local address 127.11.158.129:54893)
0926 16:37:04.596476 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:04.596533 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:04.597392 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:04.597974 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:04.597981 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:04.598359 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:04.598902 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:04.598922 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.599863 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.599868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:04.601137 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:04.601149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.601692 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.601699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:04.601949 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:04.602693 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:04.602719 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:04.604489 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:04.606360 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:04.606366 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:04.606383 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:04.606663 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:04.606967 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:04.606971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:04.606973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:04.607130 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:04.607536 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:04.607544 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:04.607835 (+ 291us) client_negotiation.cc:769] Sending connection context
0926 16:37:04.608029 (+ 194us) client_negotiation.cc:240] Negotiation successful
0926 16:37:04.608426 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1157,"thread_start_us":115,"threads_started":1}
I20260926 16:37:04.610670 12141 heartbeater.cc:347] Connected to a master server at 127.11.158.190:46703
I20260926 16:37:04.610967 12141 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:37:04.611640 12141 heartbeater.cc:511] Master 127.11.158.190:46703 requested a full tablet report, sending...
I20260926 16:37:04.613416 11950 ts_manager.cc:194] Registered new tserver with Master: ae57ae341fe54302b6fef4d5e187f46f (127.11.158.129:38185)
I20260926 16:37:04.614552 11950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:54893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:04.628753 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46703
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:33493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:04.739919 12146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:04.740245 12146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:04.740324 12146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:04.744729 12146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:04.744813 12146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:04.744904 12146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:04.750392 12146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:33493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:04.751500 12146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:04.752636 12146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:04.762405 12151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:04.762414 12154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:04.762504 12146 server_base.cc:1034] running on GCE node
W20260926 16:37:04.762424 12152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:04.763144 12146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:04.763863 12146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:04.765049 12146 hybrid_clock.cc:648] HybridClock initialized: now 1790440624765013 us; error 45 us; skew 500 ppm
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:04.767942 12146 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:04.768918 12146 webserver.cc:460] Webserver started at http://127.11.158.130:42913/ using document root <none> and password file <none>
I20260926 16:37:04.769500 12146 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:04.769611 12146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:04.769896 12146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:04.772080 12146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "03c84b6a83d948e3a11c04a5929a3abc"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "a3a43bd4353c3bdf4e246a29af5cfaf9"
server_key_iv: "0a33b7b657e711a72b9c2eb2986377a5"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.772699 12146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "03c84b6a83d948e3a11c04a5929a3abc"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "a3a43bd4353c3bdf4e246a29af5cfaf9"
server_key_iv: "0a33b7b657e711a72b9c2eb2986377a5"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.776433 12146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:37:04.779028 12161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:04.780469 12146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:04.780611 12146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "03c84b6a83d948e3a11c04a5929a3abc"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "a3a43bd4353c3bdf4e246a29af5cfaf9"
server_key_iv: "0a33b7b657e711a72b9c2eb2986377a5"
server_key_version: "encryptionkey@0"
I20260926 16:37:04.780735 12146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:04.804738 12146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:04.808548 12146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:04.808791 12146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:04.809497 12146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:04.810426 12146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:04.810524 12146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:04.810632 12146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:04.810688 12146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:04.820278 12146 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:38465
I20260926 16:37:04.820300 12274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:38465 every 8 connection(s)
I20260926 16:37:04.821262 12146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:04.825397 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12146
I20260926 16:37:04.825526 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:04.825851 11898 external_mini_cluster.cc:1428] Setting key 898e11fe1f1611f5640e40038576d0d3
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:37:04.836990 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:04.823212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:32833 (local address 127.11.158.190:46703)
0926 16:37:04.823389 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:04.823394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:04.824565 (+ 1171us) server_negotiation.cc:407] Connection header received
0926 16:37:04.825688 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:04.825691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:04.825740 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:04.825821 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:04.827389 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.827902 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.829562 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.829986 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.830129 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:04.832936 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:04.832944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:04.832950 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:04.832989 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:04.834782 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:04.835337 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:04.835342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:04.835344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:04.835440 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:04.835814 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:04.835817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:04.835819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:04.836083 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:04.836191 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:04.836754 (+ 563us) server_negotiation.cc:299] Negotiation successful
0926 16:37:04.836861 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 16:37:04.837894 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:04.823600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:46703 (local address 127.11.158.130:32833)
0926 16:37:04.824324 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:04.824443 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:04.825392 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:04.826053 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:04.826063 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:04.826584 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:04.827228 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:04.827245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.828038 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.828043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:04.829417 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:04.829428 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:04.830198 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:04.830201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:04.830358 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:04.831058 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:04.831087 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:04.832730 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:04.834917 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:04.834923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:04.834937 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:04.835209 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:04.835542 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:04.835545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:04.835546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:04.835687 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:04.836196 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:04.836204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:04.836514 (+ 310us) client_negotiation.cc:769] Sending connection context
0926 16:37:04.836676 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 16:37:04.836975 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:04.839390 12275 heartbeater.cc:347] Connected to a master server at 127.11.158.190:46703
I20260926 16:37:04.839689 12275 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:04.840312 12275 heartbeater.cc:511] Master 127.11.158.190:46703 requested a full tablet report, sending...
I20260926 16:37:04.841596 11950 ts_manager.cc:194] Registered new tserver with Master: 03c84b6a83d948e3a11c04a5929a3abc (127.11.158.130:38465)
I20260926 16:37:04.842514 11950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:32833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:04.855391 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46703
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:33493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:04.968369 12280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:04.968655 12280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:04.968700 12280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:04.972972 12280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:04.973047 12280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:04.973109 12280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:04.978458 12280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:33493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:04.979697 12280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:04.980782 12280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:04.990895 12288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:04.991092 12280 server_base.cc:1034] running on GCE node
W20260926 16:37:04.991106 12286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:04.991182 12285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:04.991557 12280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:04.992267 12280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:04.993456 12280 hybrid_clock.cc:648] HybridClock initialized: now 1790440624993416 us; error 54 us; skew 500 ppm
Sep 26 16:37:04 dist-test-slave-p39x krb5kdc[11905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:04.996173 12280 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:04.997202 12280 webserver.cc:460] Webserver started at http://127.11.158.131:42313/ using document root <none> and password file <none>
I20260926 16:37:04.997869 12280 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:04.997962 12280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:04.998208 12280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:05.000306 12280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "6837c3b7ad6041d0bd9f1761fec79e9e"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "a8cf286b0cffd7405c41ba46b56fc91f"
server_key_iv: "9fbb72fdcac7f5c0893ce56da982a32a"
server_key_version: "encryptionkey@0"
I20260926 16:37:05.000942 12280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "6837c3b7ad6041d0bd9f1761fec79e9e"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "a8cf286b0cffd7405c41ba46b56fc91f"
server_key_iv: "9fbb72fdcac7f5c0893ce56da982a32a"
server_key_version: "encryptionkey@0"
I20260926 16:37:05.004832 12280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 16:37:05.007432 12295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:05.009061 12280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:05.009222 12280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "6837c3b7ad6041d0bd9f1761fec79e9e"
format_stamp: "Formatted at 2026-09-26 16:37:04 on dist-test-slave-p39x"
server_key: "a8cf286b0cffd7405c41ba46b56fc91f"
server_key_iv: "9fbb72fdcac7f5c0893ce56da982a32a"
server_key_version: "encryptionkey@0"
I20260926 16:37:05.009357 12280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:05.029136 12280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:05.032155 12280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:05.032423 12280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:05.033046 12280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:05.033891 12280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:05.033963 12280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:05.034037 12280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:05.034080 12280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:05.043352 12280 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:37021
I20260926 16:37:05.043368 12408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:37021 every 8 connection(s)
I20260926 16:37:05.044356 12280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:05.052001 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12280
I20260926 16:37:05.052139 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:05.052553 11898 external_mini_cluster.cc:1428] Setting key 82e5024126d5fd6a766b906c9f45e335
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[11905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:05.058338 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:05.046224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:56569 (local address 127.11.158.190:46703)
0926 16:37:05.046369 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:05.046373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:05.047467 (+ 1094us) server_negotiation.cc:407] Connection header received
0926 16:37:05.048396 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:05.048400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:05.048460 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:05.048540 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:05.049696 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.050231 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.051315 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.051520 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.051595 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:05.054762 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:05.054767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:05.054770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:05.054798 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:05.056390 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:05.056906 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:05.056909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:05.056910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:05.057004 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:05.057335 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:05.057337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:05.057339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:05.057522 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:05.057618 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:05.058113 (+ 495us) server_negotiation.cc:299] Negotiation successful
0926 16:37:05.058219 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:37:05.059193 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:05.046646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:46703 (local address 127.11.158.131:56569)
0926 16:37:05.047316 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:05.047358 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:05.048105 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:05.048702 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:05.048709 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:05.049063 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:05.049564 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:05.049578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.050337 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.050339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:05.051213 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:05.051221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.051646 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.051648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:05.051870 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:05.052771 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:05.052790 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:05.054598 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:05.056493 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:05.056498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:05.056511 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:05.056784 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:05.057095 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:05.057098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:05.057099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:05.057250 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:05.057625 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:05.057633 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:05.057877 (+ 244us) client_negotiation.cc:769] Sending connection context
0926 16:37:05.058035 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 16:37:05.058313 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":119,"threads_started":1}
I20260926 16:37:05.060451 12409 heartbeater.cc:347] Connected to a master server at 127.11.158.190:46703
I20260926 16:37:05.060721 12409 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:05.061249 12409 heartbeater.cc:511] Master 127.11.158.190:46703 requested a full tablet report, sending...
I20260926 16:37:05.062366 11950 ts_manager.cc:194] Registered new tserver with Master: 6837c3b7ad6041d0bd9f1761fec79e9e (127.11.158.131:37021)
I20260926 16:37:05.062933 11950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:56569
I20260926 16:37:05.067128 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[11905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[11905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:05.093453 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:05.083447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.11.158.129:38185)
0926 16:37:05.083793 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:05.083808 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:05.083834 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:37:05.083940 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:05.083945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:05.084144 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:05.084377 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:05.085023 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.085870 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.086695 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.087067 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.087154 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:05.089861 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:05.089872 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:05.089886 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:05.089915 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:05.091710 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:05.092146 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:05.092151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:05.092153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:05.092247 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:05.092582 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:05.092584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:05.092586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:05.092835 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:05.093024 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:05.093161 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 16:37:05.093289 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":141,"threads_started":1}
W20260926 16:37:05.094609 12055 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49040
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[11905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[11905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:05.120893 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:05.112619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.11.158.129:38185)
0926 16:37:05.112786 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:05.112789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:05.112911 (+ 122us) server_negotiation.cc:407] Connection header received
0926 16:37:05.113092 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:05.113097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:05.113190 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:05.113285 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:05.114016 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.114599 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.115384 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:05.115641 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:05.115720 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:05.118073 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:05.118077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:05.118080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:05.118114 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:05.119347 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:05.119731 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:05.119733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:05.119734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:05.119806 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:05.120040 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:05.120042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:05.120043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:05.120199 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:05.120293 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:05.120574 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 16:37:05.120678 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 16:37:05.121928 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12012
I20260926 16:37:05.127785 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12146
I20260926 16:37:05.135386 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12280
I20260926 16:37:05.141031 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 11918
2026-09-26T16:37:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2197 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[12423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[12423](info): set up 2 sockets
Sep 26 16:37:05 dist-test-slave-p39x krb5kdc[12423](info): commencing operation
krb5kdc: starting...
W20260926 16:37:06.177084 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:06 dist-test-slave-p39x krb5kdc[12423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:06.304306 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:44307
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:44307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:06.412396 12436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:06.412649 12436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:06.412688 12436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:06.416571 12436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:06.416616 12436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:06.416635 12436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:06.416652 12436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:06.416667 12436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:06.421772 12436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:44307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:44307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:06.422680 12436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:06.423655 12436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:06.431879 12442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:06.431982 12441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:06.432029 12436 server_base.cc:1034] running on GCE node
W20260926 16:37:06.431991 12444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:06.432798 12436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:06.433852 12436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:06.435006 12436 hybrid_clock.cc:648] HybridClock initialized: now 1790440626434989 us; error 30 us; skew 500 ppm
Sep 26 16:37:06 dist-test-slave-p39x krb5kdc[12423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:06.437748 12436 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:06.438694 12436 webserver.cc:460] Webserver started at http://127.11.158.190:39415/ using document root <none> and password file <none>
I20260926 16:37:06.439275 12436 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:06.439373 12436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:06.439623 12436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:06.441781 12436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "594def1ef1434e2a89c6f66d7d1a6f5d"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "7c459e0ac2f21dcf4ac6f618a9545c15"
server_key_iv: "d3588b5d7a0b1a4e3b5e4fe05f82ab66"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.442380 12436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "594def1ef1434e2a89c6f66d7d1a6f5d"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "7c459e0ac2f21dcf4ac6f618a9545c15"
server_key_iv: "d3588b5d7a0b1a4e3b5e4fe05f82ab66"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.446267 12436 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 16:37:06.448776 12451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:06.449968 12436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:06.450130 12436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "594def1ef1434e2a89c6f66d7d1a6f5d"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "7c459e0ac2f21dcf4ac6f618a9545c15"
server_key_iv: "d3588b5d7a0b1a4e3b5e4fe05f82ab66"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.450251 12436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:06.462153 12436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:06.465448 12436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:06.465653 12436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:06.472718 12436 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:44307
I20260926 16:37:06.472834 12503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:44307 every 8 connection(s)
I20260926 16:37:06.473994 12436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:06.476780 12504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:06.480255 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12436
I20260926 16:37:06.480396 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:06.480657 11898 external_mini_cluster.cc:1428] Setting key 566fb420e8d837e560ecdc32837e763f
I20260926 16:37:06.482599 12504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d: Bootstrap starting.
I20260926 16:37:06.484779 12504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:06.485483 12504 log.cc:824] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:06.487450 12504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d: No bootstrap required, opened a new log
Sep 26 16:37:06 dist-test-slave-p39x krb5kdc[12423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:06.492161 12504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 44307 } }
I20260926 16:37:06.492493 12504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:06.492556 12504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594def1ef1434e2a89c6f66d7d1a6f5d, State: Initialized, Role: FOLLOWER
I20260926 16:37:06.493039 12504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 44307 } }
I20260926 16:37:06.493217 12504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:06.493278 12504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:06.493382 12504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:06.494311 12504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 44307 } }
I20260926 16:37:06.494644 12504 leader_election.cc:302] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 594def1ef1434e2a89c6f66d7d1a6f5d; no voters:
I20260926 16:37:06.494980 12504 leader_election.cc:288] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:06.495105 12509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:06.495281 12509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [term 1 LEADER]: Becoming Leader. State: Replica: 594def1ef1434e2a89c6f66d7d1a6f5d, State: Running, Role: LEADER
I20260926 16:37:06.495322 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:06.482047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.11.158.190:44307)
0926 16:37:06.482475 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:06.482490 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:06.482524 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:37:06.483203 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:06.483218 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:06.483560 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:06.483914 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:06.484707 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.485808 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.486966 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.487417 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.487584 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:06.490230 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:06.490237 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:06.490259 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:06.490296 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:06.492246 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:06.492808 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:06.492813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:06.492819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:06.492950 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:06.493229 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:06.493239 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:06.493241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:06.493614 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:06.493831 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:06.494120 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 16:37:06.494386 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
I20260926 16:37:06.495579 12509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 44307 } }
I20260926 16:37:06.496218 12504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:06.496730 12510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 594def1ef1434e2a89c6f66d7d1a6f5d. Latest consensus state: current_term: 1 leader_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 44307 } } }
I20260926 16:37:06.496840 12510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:06.497113 12511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594def1ef1434e2a89c6f66d7d1a6f5d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 44307 } } }
I20260926 16:37:06.497201 12511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:06.497994 12514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:06.501271 12514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:06.506662 12514 catalog_manager.cc:1347] Generated new cluster ID: af8a2d236f7a4b21ae0c75e511f7dcc2
I20260926 16:37:06.506735 12514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:06.517485 12514 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:06.518507 12514 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:06.535846 12514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 594def1ef1434e2a89c6f66d7d1a6f5d: Generated new TSK 0
I20260926 16:37:06.536804 12514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:06.575834 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:44307
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:06.686553 12530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:06.686832 12530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:06.686879 12530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:06.690953 12530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:06.691062 12530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:06.691162 12530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:06.696625 12530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:44307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:06.697813 12530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:06.698917 12530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:06.708557 12535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:06.708540 12536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:06.708540 12538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:06.708930 12530 server_base.cc:1034] running on GCE node
I20260926 16:37:06.709311 12530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:06.709889 12530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:06.711069 12530 hybrid_clock.cc:648] HybridClock initialized: now 1790440626711053 us; error 27 us; skew 500 ppm
Sep 26 16:37:06 dist-test-slave-p39x krb5kdc[12423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:06.713848 12530 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:06.714699 12530 webserver.cc:460] Webserver started at http://127.11.158.129:45913/ using document root <none> and password file <none>
I20260926 16:37:06.715240 12530 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:06.715297 12530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:06.715462 12530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:06.717537 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "85e1beba0b66488b9a46beaac740f97a"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "26795ac7f1add984d928838da819b73f"
server_key_iv: "074174e0859a19824e158764b361afe5"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.718056 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "85e1beba0b66488b9a46beaac740f97a"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "26795ac7f1add984d928838da819b73f"
server_key_iv: "074174e0859a19824e158764b361afe5"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.721740 12530 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:37:06.724062 12545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:06.725327 12530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:06.725438 12530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "85e1beba0b66488b9a46beaac740f97a"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "26795ac7f1add984d928838da819b73f"
server_key_iv: "074174e0859a19824e158764b361afe5"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.725528 12530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:06.755752 12530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:06.758845 12530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:06.759002 12530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:06.759550 12530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:06.760360 12530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:06.760406 12530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:06.760444 12530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:06.760466 12530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:06.769510 12530 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:34233
I20260926 16:37:06.769533 12658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:34233 every 8 connection(s)
I20260926 16:37:06.770504 12530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:06.772485 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12530
I20260926 16:37:06.772590 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:06.772853 11898 external_mini_cluster.cc:1428] Setting key 0c5370eddb87f3aef302a9a782339d15
Sep 26 16:37:06 dist-test-slave-p39x krb5kdc[12423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:37:06.785801 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:06.772631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:52127 (local address 127.11.158.190:44307)
0926 16:37:06.772753 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:06.772757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:06.773879 (+ 1122us) server_negotiation.cc:407] Connection header received
0926 16:37:06.775717 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:06.775720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:06.775768 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:06.775832 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:06.777193 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.777730 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.778988 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.779235 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.779347 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:06.781920 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:06.781925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:06.781931 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:06.781986 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:06.783415 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:06.784033 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:06.784036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:06.784037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:06.784112 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:06.784563 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:06.784570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:06.784574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:06.784801 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:06.784882 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:06.785543 (+ 661us) server_negotiation.cc:299] Negotiation successful
0926 16:37:06.785653 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 16:37:06.787078 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:06.773164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:44307 (local address 127.11.158.129:52127)
0926 16:37:06.773702 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:06.773760 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:06.775468 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:06.776055 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:06.776066 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:06.776544 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:06.777037 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:06.777050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.777864 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.777868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:06.778857 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:06.778867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.779385 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.779390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:06.779570 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:06.780206 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:06.780222 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:06.781716 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:06.783545 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:06.783553 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:06.783592 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:06.783943 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:06.784224 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:06.784231 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:06.784235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:06.784456 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:06.784914 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:06.784925 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:06.785273 (+ 348us) client_negotiation.cc:769] Sending connection context
0926 16:37:06.785485 (+ 212us) client_negotiation.cc:240] Negotiation successful
0926 16:37:06.785879 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":118,"threads_started":1}
I20260926 16:37:06.788583 12659 heartbeater.cc:347] Connected to a master server at 127.11.158.190:44307
I20260926 16:37:06.788846 12659 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:06.789361 12659 heartbeater.cc:511] Master 127.11.158.190:44307 requested a full tablet report, sending...
I20260926 16:37:06.790900 12468 ts_manager.cc:194] Registered new tserver with Master: 85e1beba0b66488b9a46beaac740f97a (127.11.158.129:34233)
I20260926 16:37:06.792012 12468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:52127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:06.799569 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:44307
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:06.912454 12664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:06.912750 12664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:06.912797 12664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:06.916966 12664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:06.917038 12664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:06.917102 12664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:06.922621 12664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:44307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:06.923810 12664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:06.924968 12664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:06.935233 12669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:06.935433 12672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:06.935290 12670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:06.935535 12664 server_base.cc:1034] running on GCE node
I20260926 16:37:06.936025 12664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:06.936784 12664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:06.937989 12664 hybrid_clock.cc:648] HybridClock initialized: now 1790440626937971 us; error 30 us; skew 500 ppm
Sep 26 16:37:06 dist-test-slave-p39x krb5kdc[12423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:06.940845 12664 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:06.941686 12664 webserver.cc:460] Webserver started at http://127.11.158.130:39417/ using document root <none> and password file <none>
I20260926 16:37:06.942219 12664 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:06.942276 12664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:06.942443 12664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:06.944494 12664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "688586b2a3c1475eb3f419fe37bda260"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "f11cf7fbdc94ebc3a660fe6ce8a46200"
server_key_iv: "db08a6fbcce8efba81d4859d52b9e9d7"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.945097 12664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "688586b2a3c1475eb3f419fe37bda260"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "f11cf7fbdc94ebc3a660fe6ce8a46200"
server_key_iv: "db08a6fbcce8efba81d4859d52b9e9d7"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.948907 12664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:37:06.951354 12679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:06.952673 12664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:06.952803 12664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "688586b2a3c1475eb3f419fe37bda260"
format_stamp: "Formatted at 2026-09-26 16:37:06 on dist-test-slave-p39x"
server_key: "f11cf7fbdc94ebc3a660fe6ce8a46200"
server_key_iv: "db08a6fbcce8efba81d4859d52b9e9d7"
server_key_version: "encryptionkey@0"
I20260926 16:37:06.952911 12664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:06.965173 12664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:06.968148 12664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:06.968407 12664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:06.969007 12664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:06.969830 12664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:06.969875 12664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:06.969913 12664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:06.969952 12664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:06.978657 12664 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:34983
I20260926 16:37:06.978681 12792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:34983 every 8 connection(s)
I20260926 16:37:06.979584 12664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:06.985963 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12664
I20260926 16:37:06.986060 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:06.986341 11898 external_mini_cluster.cc:1428] Setting key db36ddd1f6bec1e98c4ad446c28e482a
Sep 26 16:37:06 dist-test-slave-p39x krb5kdc[12423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:06.993907 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:06.981436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:39449 (local address 127.11.158.190:44307)
0926 16:37:06.981607 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:06.981610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:06.982402 (+ 792us) server_negotiation.cc:407] Connection header received
0926 16:37:06.983731 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:06.983735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:06.983793 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:06.983907 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:06.985163 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.985715 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.987195 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.987414 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.987548 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:06.990152 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:06.990155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:06.990157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:06.990181 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:06.991755 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:06.992306 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:06.992312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:06.992313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:06.992450 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:06.992797 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:06.992801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:06.992803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:06.992992 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:06.993120 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:06.993612 (+ 492us) server_negotiation.cc:299] Negotiation successful
0926 16:37:06.993738 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 16:37:06.994736 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:06.981804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:44307 (local address 127.11.158.130:39449)
0926 16:37:06.982251 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:06.982293 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:06.983517 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:06.984086 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:06.984092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:06.984523 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:06.985015 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:06.985028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.985847 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.985851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:06.987062 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:06.987072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:06.987597 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:06.987601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:06.987768 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:06.988432 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:06.988449 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:06.989968 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:06.991885 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:06.991891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:06.991906 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:06.992179 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:06.992544 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:06.992557 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:06.992559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:06.992703 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:06.993111 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:06.993118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:06.993369 (+ 251us) client_negotiation.cc:769] Sending connection context
0926 16:37:06.993565 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 16:37:06.993895 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:37:06.995927 12793 heartbeater.cc:347] Connected to a master server at 127.11.158.190:44307
I20260926 16:37:06.996151 12793 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:06.996737 12793 heartbeater.cc:511] Master 127.11.158.190:44307 requested a full tablet report, sending...
I20260926 16:37:06.997794 12468 ts_manager.cc:194] Registered new tserver with Master: 688586b2a3c1475eb3f419fe37bda260 (127.11.158.130:34983)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:06.998472 12468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:39449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:07.015686 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:44307
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:07.127955 12798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:07.128237 12798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:07.128314 12798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:07.132808 12798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:07.132874 12798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:07.132961 12798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:07.138388 12798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:44307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:07.139521 12798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:07.140641 12798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:07.150028 12803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:07.150041 12798 server_base.cc:1034] running on GCE node
W20260926 16:37:07.149951 12804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:07.149951 12806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:07.150687 12798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:07.151405 12798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:07.152594 12798 hybrid_clock.cc:648] HybridClock initialized: now 1790440627152566 us; error 55 us; skew 500 ppm
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:07.155783 12798 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:07.156821 12798 webserver.cc:460] Webserver started at http://127.11.158.131:42941/ using document root <none> and password file <none>
I20260926 16:37:07.157428 12798 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:07.157492 12798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:07.157665 12798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:07.159696 12798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "38fcd5cf56e34e55ac86faacd0220b5a"
format_stamp: "Formatted at 2026-09-26 16:37:07 on dist-test-slave-p39x"
server_key: "138462dd4ba1fb307131f323c51c7d2d"
server_key_iv: "d0f1b6a23f078c08d4d46af0f0762b74"
server_key_version: "encryptionkey@0"
I20260926 16:37:07.160231 12798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "38fcd5cf56e34e55ac86faacd0220b5a"
format_stamp: "Formatted at 2026-09-26 16:37:07 on dist-test-slave-p39x"
server_key: "138462dd4ba1fb307131f323c51c7d2d"
server_key_iv: "d0f1b6a23f078c08d4d46af0f0762b74"
server_key_version: "encryptionkey@0"
I20260926 16:37:07.164212 12798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 16:37:07.166872 12813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:07.168268 12798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:07.168452 12798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "38fcd5cf56e34e55ac86faacd0220b5a"
format_stamp: "Formatted at 2026-09-26 16:37:07 on dist-test-slave-p39x"
server_key: "138462dd4ba1fb307131f323c51c7d2d"
server_key_iv: "d0f1b6a23f078c08d4d46af0f0762b74"
server_key_version: "encryptionkey@0"
I20260926 16:37:07.168560 12798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:07.186367 12798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:07.189929 12798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:07.190140 12798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:07.190748 12798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:07.191617 12798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:07.191687 12798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:07.191754 12798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:07.191790 12798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:07.200544 12798 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:36063
I20260926 16:37:07.200577 12926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:36063 every 8 connection(s)
I20260926 16:37:07.201481 12798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:07.202147 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12798
I20260926 16:37:07.202278 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:07.202554 11898 external_mini_cluster.cc:1428] Setting key 39ae48f7618bd11a5b1bd909ef365707
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440627, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:07.214950 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.203560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:52875 (local address 127.11.158.190:44307)
0926 16:37:07.203690 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.203694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.204736 (+ 1042us) server_negotiation.cc:407] Connection header received
0926 16:37:07.206022 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.206024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.206069 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.206139 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:07.207279 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.207804 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.209057 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.209276 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.209354 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.211744 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:07.211747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:07.211749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:07.211776 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:07.213086 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.213605 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.213608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.213610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.213686 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.213978 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.213980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.213981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.214162 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:07.214237 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.214729 (+ 492us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.214826 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 16:37:07.216109 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.203978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:44307 (local address 127.11.158.131:52875)
0926 16:37:07.204595 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:07.204657 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:07.205832 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:07.206296 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:07.206303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:07.206648 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:07.207149 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.207162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.207956 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.207959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.208953 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.208963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.209401 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.209404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.209544 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.210125 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:07.210141 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:07.211586 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:07.213223 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:07.213228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:07.213240 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:07.213511 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:07.213766 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:07.213769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:07.213770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:07.213901 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:07.214285 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:07.214290 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:07.214520 (+ 230us) client_negotiation.cc:769] Sending connection context
0926 16:37:07.214687 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 16:37:07.215032 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":135,"threads_started":1}
I20260926 16:37:07.217135 12927 heartbeater.cc:347] Connected to a master server at 127.11.158.190:44307
I20260926 16:37:07.217386 12927 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:07.217908 12927 heartbeater.cc:511] Master 127.11.158.190:44307 requested a full tablet report, sending...
I20260926 16:37:07.218950 12468 ts_manager.cc:194] Registered new tserver with Master: 38fcd5cf56e34e55ac86faacd0220b5a (127.11.158.131:36063)
I20260926 16:37:07.219527 12468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:52875
I20260926 16:37:07.228745 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:37:07.240120 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.232466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.11.158.190:44307)
0926 16:37:07.232634 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.232638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.232807 (+ 169us) server_negotiation.cc:407] Connection header received
0926 16:37:07.232885 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.232888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.232935 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.233019 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:07.233733 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.234263 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.235166 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.235384 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.235472 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.236999 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:07.237002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:07.237005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:07.237032 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:07.238454 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.238900 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.238903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.238904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.238989 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.239259 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.239262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.239263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.239501 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:07.239644 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.239883 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.239988 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 16:37:07.245339 12468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:07.248495 12468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:07.264068 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.257624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:34983 (local address 127.0.0.1:51568)
0926 16:37:07.258092 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:07.258161 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:07.258324 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:07.259318 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:07.259325 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:07.259350 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:07.259505 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.259512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.261085 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.261089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.262460 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.262469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.263366 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.263369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.263576 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.263639 (+ 63us) client_negotiation.cc:769] Sending connection context
0926 16:37:07.263766 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 16:37:07.263900 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":50,"thread_start_us":100,"threads_started":1}
I20260926 16:37:07.264068 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.257765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.11.158.130:34983)
0926 16:37:07.258073 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.258086 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.258251 (+ 165us) server_negotiation.cc:407] Connection header received
0926 16:37:07.258668 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.258676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.258903 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.259067 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:07.259780 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.260924 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.262582 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.263236 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.263385 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.263539 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.263813 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.263913 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
I20260926 16:37:07.265111 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.258800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:36063 (local address 127.0.0.1:47114)
0926 16:37:07.259070 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:07.259084 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:07.259172 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:07.260555 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:07.260558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:07.260576 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:07.260673 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.260678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.262000 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.262003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.263129 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.263146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.264693 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.264697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.264801 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.264826 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:37:07.264875 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 16:37:07.264948 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
I20260926 16:37:07.265350 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.259300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.11.158.129:34233)
0926 16:37:07.259593 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.259606 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.260192 (+ 586us) server_negotiation.cc:407] Connection header received
0926 16:37:07.260276 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.260283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.260481 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.260633 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:07.261040 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.262216 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.263663 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.264502 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.264639 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.264775 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.265064 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.265193 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260926 16:37:07.265784 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.259149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34233 (local address 127.0.0.1:43702)
0926 16:37:07.260053 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:07.260068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:07.260177 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:07.260798 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:07.260802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:07.260819 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:07.260908 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.260914 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.262361 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.262372 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.263555 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.263564 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.264829 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.264832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.264922 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.264940 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:37:07.265575 (+ 635us) client_negotiation.cc:240] Negotiation successful
0926 16:37:07.265655 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":42,"thread_start_us":88,"threads_started":1}
I20260926 16:37:07.266083 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.259633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.11.158.131:36063)
0926 16:37:07.259958 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.259966 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.259985 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:37:07.260045 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.260050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.260410 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.260604 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:07.260757 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.261811 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.263299 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.264552 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.265556 (+ 1004us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.265694 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.265743 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.265938 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":151,"threads_started":1}
I20260926 16:37:07.267535 12727 tablet_service.cc:1467] Processing CreateTablet for tablet 357aee5708c640cf9a5d314e9b6bdbd6 (DEFAULT_TABLE table=test-table [id=6eb304e28b9c43c285ec020895b7b872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:07.268805 12727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357aee5708c640cf9a5d314e9b6bdbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:07.268714 12593 tablet_service.cc:1467] Processing CreateTablet for tablet 357aee5708c640cf9a5d314e9b6bdbd6 (DEFAULT_TABLE table=test-table [id=6eb304e28b9c43c285ec020895b7b872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:07.268951 12861 tablet_service.cc:1467] Processing CreateTablet for tablet 357aee5708c640cf9a5d314e9b6bdbd6 (DEFAULT_TABLE table=test-table [id=6eb304e28b9c43c285ec020895b7b872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:07.269786 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357aee5708c640cf9a5d314e9b6bdbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:07.269927 12861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357aee5708c640cf9a5d314e9b6bdbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:07.276172 12944 tablet_bootstrap.cc:492] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a: Bootstrap starting.
I20260926 16:37:07.277667 12945 tablet_bootstrap.cc:492] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260: Bootstrap starting.
I20260926 16:37:07.278945 12944 tablet_bootstrap.cc:654] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:07.279239 12946 tablet_bootstrap.cc:492] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a: Bootstrap starting.
I20260926 16:37:07.280156 12945 tablet_bootstrap.cc:654] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:07.280717 12944 log.cc:824] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:07.280958 12945 log.cc:824] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:07.281373 12946 tablet_bootstrap.cc:654] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:07.282522 12946 log.cc:824] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:07.283226 12945 tablet_bootstrap.cc:492] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260: No bootstrap required, opened a new log
I20260926 16:37:07.283478 12945 ts_tablet_manager.cc:1397] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 16:37:07.284617 12946 tablet_bootstrap.cc:492] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a: No bootstrap required, opened a new log
I20260926 16:37:07.284821 12946 ts_tablet_manager.cc:1397] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:37:07.285646 12944 tablet_bootstrap.cc:492] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a: No bootstrap required, opened a new log
I20260926 16:37:07.285823 12944 ts_tablet_manager.cc:1397] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260926 16:37:07.287437 12945 raft_consensus.cc:348] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.287770 12945 raft_consensus.cc:374] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:07.287835 12945 raft_consensus.cc:729] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688586b2a3c1475eb3f419fe37bda260, State: Initialized, Role: FOLLOWER
I20260926 16:37:07.287770 12946 raft_consensus.cc:348] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.288053 12946 raft_consensus.cc:374] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:07.288120 12946 raft_consensus.cc:729] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e1beba0b66488b9a46beaac740f97a, State: Initialized, Role: FOLLOWER
I20260926 16:37:07.288372 12945 consensus_queue.cc:260] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.288630 12946 consensus_queue.cc:260] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.289158 12793 heartbeater.cc:503] Master 127.11.158.190:44307 was elected leader, sending a full tablet report...
I20260926 16:37:07.289441 12945 ts_tablet_manager.cc:1428] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:37:07.289558 12944 raft_consensus.cc:348] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.289830 12944 raft_consensus.cc:374] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:07.289888 12944 raft_consensus.cc:729] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38fcd5cf56e34e55ac86faacd0220b5a, State: Initialized, Role: FOLLOWER
I20260926 16:37:07.290365 12946 ts_tablet_manager.cc:1428] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:37:07.290303 12944 consensus_queue.cc:260] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.290692 12659 heartbeater.cc:503] Master 127.11.158.190:44307 was elected leader, sending a full tablet report...
I20260926 16:37:07.291188 12944 ts_tablet_manager.cc:1428] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 16:37:07.291550 12927 heartbeater.cc:503] Master 127.11.158.190:44307 was elected leader, sending a full tablet report...
I20260926 16:37:07.376938 12952 raft_consensus.cc:482] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:07.377202 12952 raft_consensus.cc:504] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.378459 12952 leader_election.cc:288] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 688586b2a3c1475eb3f419fe37bda260 (127.11.158.130:34983), 85e1beba0b66488b9a46beaac740f97a (127.11.158.129:34233)
I20260926 16:37:07.382679 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.378667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:34983 (local address 127.11.158.131:35141)
0926 16:37:07.378793 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:07.378811 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:07.378911 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:07.379240 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:07.379243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:07.379325 (+ 82us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:07.379465 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.379472 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.380437 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.380441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.381270 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.381289 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.382200 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.382204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.382314 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.382335 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:37:07.382420 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:37:07.382519 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":16,"spinlock_wait_cycles":10880}
I20260926 16:37:07.382678 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.378805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:35141 (local address 127.11.158.130:34983)
0926 16:37:07.378981 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.378988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.379003 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:37:07.379061 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.379064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.379115 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.379201 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:07.379634 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.380317 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.381419 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.382066 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.382194 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.382248 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.382436 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.382512 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 16:37:07.382848 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.378957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:54947 (local address 127.11.158.129:34233)
0926 16:37:07.379087 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.379091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.379293 (+ 202us) server_negotiation.cc:407] Connection header received
0926 16:37:07.379433 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.379438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.379489 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.379568 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:07.379903 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.380452 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.381592 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.382100 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.382226 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.382278 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.382627 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.382722 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:37:07.383284 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.378781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34233 (local address 127.11.158.131:54947)
0926 16:37:07.379191 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:07.379204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:07.379326 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:07.379634 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:07.379638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:07.379658 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:07.379780 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.379787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.380568 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.380571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.381456 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:07.381466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.382333 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.382339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:07.382458 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.382482 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:37:07.383027 (+ 545us) client_negotiation.cc:240] Negotiation successful
0926 16:37:07.383128 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"spinlock_wait_cycles":44288,"thread_start_us":119,"threads_started":1}
I20260926 16:37:07.383462 12747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357aee5708c640cf9a5d314e9b6bdbd6" candidate_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688586b2a3c1475eb3f419fe37bda260" is_pre_election: true
I20260926 16:37:07.383852 12747 raft_consensus.cc:2454] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38fcd5cf56e34e55ac86faacd0220b5a in term 0.
I20260926 16:37:07.383919 12613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357aee5708c640cf9a5d314e9b6bdbd6" candidate_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e1beba0b66488b9a46beaac740f97a" is_pre_election: true
I20260926 16:37:07.384223 12613 raft_consensus.cc:2454] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38fcd5cf56e34e55ac86faacd0220b5a in term 0.
I20260926 16:37:07.384442 12816 leader_election.cc:302] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38fcd5cf56e34e55ac86faacd0220b5a, 688586b2a3c1475eb3f419fe37bda260; no voters:
I20260926 16:37:07.384779 12952 raft_consensus.cc:2781] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:07.384887 12952 raft_consensus.cc:482] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:07.384977 12952 raft_consensus.cc:3037] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:07.386132 12952 raft_consensus.cc:504] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.386518 12952 leader_election.cc:288] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [CANDIDATE]: Term 1 election: Requested vote from peers 688586b2a3c1475eb3f419fe37bda260 (127.11.158.130:34983), 85e1beba0b66488b9a46beaac740f97a (127.11.158.129:34233)
I20260926 16:37:07.387019 12613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357aee5708c640cf9a5d314e9b6bdbd6" candidate_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e1beba0b66488b9a46beaac740f97a"
I20260926 16:37:07.387017 12747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357aee5708c640cf9a5d314e9b6bdbd6" candidate_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688586b2a3c1475eb3f419fe37bda260"
I20260926 16:37:07.387161 12747 raft_consensus.cc:3037] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:07.387161 12613 raft_consensus.cc:3037] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:07.388497 12747 raft_consensus.cc:2454] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38fcd5cf56e34e55ac86faacd0220b5a in term 1.
I20260926 16:37:07.388496 12613 raft_consensus.cc:2454] T 357aee5708c640cf9a5d314e9b6bdbd6 P 85e1beba0b66488b9a46beaac740f97a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38fcd5cf56e34e55ac86faacd0220b5a in term 1.
I20260926 16:37:07.388832 12814 leader_election.cc:302] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38fcd5cf56e34e55ac86faacd0220b5a, 85e1beba0b66488b9a46beaac740f97a; no voters:
I20260926 16:37:07.389045 12952 raft_consensus.cc:2781] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:07.389312 12952 raft_consensus.cc:686] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [term 1 LEADER]: Becoming Leader. State: Replica: 38fcd5cf56e34e55ac86faacd0220b5a, State: Running, Role: LEADER
I20260926 16:37:07.389658 12952 consensus_queue.cc:237] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } }
I20260926 16:37:07.393100 12467 catalog_manager.cc:5495] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 38fcd5cf56e34e55ac86faacd0220b5a (127.11.158.131). New cstate: current_term: 1 leader_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688586b2a3c1475eb3f419fe37bda260" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38fcd5cf56e34e55ac86faacd0220b5a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 36063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e1beba0b66488b9a46beaac740f97a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34233 } health_report { overall_health: UNKNOWN } } }
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:07.422979 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.413792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.11.158.129:34233)
0926 16:37:07.413955 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.413960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.414106 (+ 146us) server_negotiation.cc:407] Connection header received
0926 16:37:07.414211 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.414213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.414258 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.414330 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:07.414980 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.415491 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.416292 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.416605 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.416696 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.419389 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:07.419403 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:07.419419 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:07.419448 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:07.421155 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.421565 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.421571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.421573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.421669 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.421967 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.421971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.421972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.422258 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:07.422427 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.422636 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.422780 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440627, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440627, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:07.451352 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.442521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.11.158.129:34233)
0926 16:37:07.442656 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.442661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.442863 (+ 202us) server_negotiation.cc:407] Connection header received
0926 16:37:07.443049 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.443052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.443097 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.443193 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:07.443888 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.444500 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.445336 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.445568 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.445680 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.448179 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:07.448182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:07.448184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:07.448219 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:07.449714 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.450143 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.450146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.450148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.450230 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.450467 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.450470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.450471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.450635 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:07.450736 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.450994 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.451151 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260926 16:37:07.452540 12928 tablet.cc:2369] T 357aee5708c640cf9a5d314e9b6bdbd6 P 38fcd5cf56e34e55ac86faacd0220b5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260926 16:37:07.480732 12794 tablet.cc:2369] T 357aee5708c640cf9a5d314e9b6bdbd6 P 688586b2a3c1475eb3f419fe37bda260: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:07.485773 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:07.469375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.11.158.129:34233)
0926 16:37:07.469516 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:07.469521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:07.469683 (+ 162us) server_negotiation.cc:407] Connection header received
0926 16:37:07.469785 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:07.469788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:07.469832 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:07.469910 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:07.470620 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.471138 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.471986 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:07.472315 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:07.472472 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:07.482381 (+ 9909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:07.482386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:07.482389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:07.482427 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:07.484000 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.484561 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.484564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.484566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.484666 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:07.484935 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:07.484938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:07.484940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:07.485142 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:07.485264 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:07.485493 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 16:37:07.485615 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 16:37:07.487182 12573 tablet_service.cc:2908] Rejecting scan request for tablet 357aee5708c640cf9a5d314e9b6bdbd6: Uninitialized: clean time has not yet been initialized
I20260926 16:37:07.489399 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12530
I20260926 16:37:07.495584 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12664
I20260926 16:37:07.501279 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12798
I20260926 16:37:07.507072 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12436
2026-09-26T16:37:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2367 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12968](info): set up 2 sockets
Sep 26 16:37:07 dist-test-slave-p39x krb5kdc[12968](info): commencing operation
krb5kdc: starting...
W20260926 16:37:08.612805 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:08 dist-test-slave-p39x krb5kdc[12968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:08.740587 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39859
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:39859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:08.851998 12981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:08.852391 12981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:08.852479 12981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:08.856659 12981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:08.856755 12981 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 16:37:08.856788 12981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:08.856832 12981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:08.856889 12981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:08.856926 12981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:08.862457 12981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:39859
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:08.863667 12981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:08.864794 12981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:08.873654 12987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:08.873710 12981 server_base.cc:1034] running on GCE node
W20260926 16:37:08.873636 12986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:08.873625 12989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:08.874465 12981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:08.875919 12981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:08.877085 12981 hybrid_clock.cc:648] HybridClock initialized: now 1790440628877056 us; error 41 us; skew 500 ppm
Sep 26 16:37:08 dist-test-slave-p39x krb5kdc[12968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:08.879848 12981 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:08.880869 12981 webserver.cc:460] Webserver started at http://127.11.158.190:38699/ using document root <none> and password file <none>
I20260926 16:37:08.881395 12981 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:08.881450 12981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:08.881621 12981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:08.883582 12981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "a6d92e49122146798e112c928dfbce33"
format_stamp: "Formatted at 2026-09-26 16:37:08 on dist-test-slave-p39x"
server_key: "61d7f8d4eac7ca9b8445c0554d6afc05"
server_key_iv: "2cec44829e9d41264d07658f58cc5d07"
server_key_version: "encryptionkey@0"
I20260926 16:37:08.884094 12981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "a6d92e49122146798e112c928dfbce33"
format_stamp: "Formatted at 2026-09-26 16:37:08 on dist-test-slave-p39x"
server_key: "61d7f8d4eac7ca9b8445c0554d6afc05"
server_key_iv: "2cec44829e9d41264d07658f58cc5d07"
server_key_version: "encryptionkey@0"
I20260926 16:37:08.887843 12981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:37:08.890372 12996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:08.891698 12981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:08.891849 12981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "a6d92e49122146798e112c928dfbce33"
format_stamp: "Formatted at 2026-09-26 16:37:08 on dist-test-slave-p39x"
server_key: "61d7f8d4eac7ca9b8445c0554d6afc05"
server_key_iv: "2cec44829e9d41264d07658f58cc5d07"
server_key_version: "encryptionkey@0"
I20260926 16:37:08.891964 12981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:08.907097 12981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:08.910343 12981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:08.910575 12981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:08.919541 12981 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:39859
I20260926 16:37:08.919559 13058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:39859 every 8 connection(s)
I20260926 16:37:08.920663 12981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:08.923458 13059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:08.927229 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 12981
I20260926 16:37:08.927347 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:08.927592 11898 external_mini_cluster.cc:1428] Setting key 4bfdd2fec0ede0b1ae6fea7f6740d62f
I20260926 16:37:08.929636 13059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: Bootstrap starting.
I20260926 16:37:08.932307 13059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:08.933079 13059 log.cc:824] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:08.935168 13059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: No bootstrap required, opened a new log
Sep 26 16:37:08 dist-test-slave-p39x krb5kdc[12968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:08.939517 13059 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d92e49122146798e112c928dfbce33" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39859 } }
I20260926 16:37:08.939785 13059 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:08.939831 13059 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d92e49122146798e112c928dfbce33, State: Initialized, Role: FOLLOWER
I20260926 16:37:08.940388 13059 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d92e49122146798e112c928dfbce33" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39859 } }
I20260926 16:37:08.940559 13059 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:08.940629 13059 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:08.940737 13059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:08.941895 13059 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d92e49122146798e112c928dfbce33" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39859 } }
I20260926 16:37:08.942258 13059 leader_election.cc:302] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6d92e49122146798e112c928dfbce33; no voters:
I20260926 16:37:08.942587 13059 leader_election.cc:288] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:08.942701 13064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:08.942971 13064 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [term 1 LEADER]: Becoming Leader. State: Replica: a6d92e49122146798e112c928dfbce33, State: Running, Role: LEADER
I20260926 16:37:08.943279 13064 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d92e49122146798e112c928dfbce33" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39859 } }
I20260926 16:37:08.943390 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:08.929121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.11.158.190:39859)
0926 16:37:08.929626 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:08.929652 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:08.929697 (+ 45us) server_negotiation.cc:407] Connection header received
0926 16:37:08.930704 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:08.930729 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:08.931091 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:08.931427 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:08.932222 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:08.933446 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:08.934602 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:08.935083 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:08.935209 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:08.937984 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:08.937992 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:08.938011 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:08.938043 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:08.940122 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:08.940678 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:08.940685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:08.940690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:08.940836 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:08.941112 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:08.941115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:08.941116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:08.941627 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:08.941828 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:08.942045 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 16:37:08.942339 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":130,"threads_started":1}
I20260926 16:37:08.943867 13059 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:08.944545 13065 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6d92e49122146798e112c928dfbce33. Latest consensus state: current_term: 1 leader_uuid: "a6d92e49122146798e112c928dfbce33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d92e49122146798e112c928dfbce33" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39859 } } }
I20260926 16:37:08.944691 13065 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:08.944975 13066 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6d92e49122146798e112c928dfbce33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d92e49122146798e112c928dfbce33" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39859 } } }
I20260926 16:37:08.945084 13066 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33 [sys.catalog]: This master's current role is: LEADER
W20260926 16:37:08.949265 13079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:37:08.949368 13079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:37:08.949436 13071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:08.950134 13071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:08.955493 13071 catalog_manager.cc:1347] Generated new cluster ID: 798e90fe662a4a6d90f7367135957a86
I20260926 16:37:08.955576 13071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:08.972983 13071 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:08.973973 13071 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:08.984494 13071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: Generated new TSK 0
I20260926 16:37:08.985289 13071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:09.021486 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39859
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:09.133759 13085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:09.134034 13085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:09.134081 13085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:09.138233 13085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:09.138300 13085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:09.138340 13085 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:09.138420 13085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:09.143836 13085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:39859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:09.145154 13085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:09.146262 13085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:09.156243 13091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:09.156234 13093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:09.156234 13090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:09.156487 13085 server_base.cc:1034] running on GCE node
I20260926 16:37:09.157061 13085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:09.157773 13085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:09.158967 13085 hybrid_clock.cc:648] HybridClock initialized: now 1790440629158941 us; error 46 us; skew 500 ppm
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:09.161933 13085 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:09.163002 13085 webserver.cc:460] Webserver started at http://127.11.158.129:35771/ using document root <none> and password file <none>
I20260926 16:37:09.163551 13085 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:09.163625 13085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:09.163795 13085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:09.166541 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "e3732e84eab84b8f9f75c33fa0eb17be"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "5798b6a7ffd5e5dd5e2873a37554ee02"
server_key_iv: "6fc7c9752e1750dc5004971ffca3c315"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.167146 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "e3732e84eab84b8f9f75c33fa0eb17be"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "5798b6a7ffd5e5dd5e2873a37554ee02"
server_key_iv: "6fc7c9752e1750dc5004971ffca3c315"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.171658 13085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:37:09.174712 13100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:09.176292 13085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:09.176483 13085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "e3732e84eab84b8f9f75c33fa0eb17be"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "5798b6a7ffd5e5dd5e2873a37554ee02"
server_key_iv: "6fc7c9752e1750dc5004971ffca3c315"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.176587 13085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:09.204288 13085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:09.207110 13085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:09.207258 13085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:09.208756 13085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:09.208806 13085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:09.208844 13085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:09.208904 13085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:09.219280 13085 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:34989
I20260926 16:37:09.219420 13215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:34989 every 8 connection(s)
I20260926 16:37:09.220467 13085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:09.223057 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.209576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:33895 (local address 127.11.158.190:39859)
0926 16:37:09.209750 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.209757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.210565 (+ 808us) server_negotiation.cc:407] Connection header received
0926 16:37:09.211760 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.211766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.211835 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.211941 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:09.213525 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.214434 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.216211 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.216644 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.216768 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.219375 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:09.219379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:09.219382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:09.219409 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:09.221094 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.221710 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.221713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.221714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.221790 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.222053 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.222055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.222056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.222228 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:09.222305 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.222827 (+ 522us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.222936 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 16:37:09.223875 13109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.209955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39859 (local address 127.11.158.129:33895)
0926 16:37:09.210395 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.210443 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.211455 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.212171 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.212181 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.212630 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:09.213351 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.213374 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.214615 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.214618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.216092 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.216107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.216794 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.216796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.216936 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.217589 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:09.217610 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:09.219189 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:09.221235 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.221242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.221260 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.221599 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.221856 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.221858 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.221859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.221977 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.222351 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:09.222357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:09.222611 (+ 254us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.222788 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.223040 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":131,"threads_started":1}
I20260926 16:37:09.225225 13216 heartbeater.cc:347] Connected to a master server at 127.11.158.190:39859
I20260926 16:37:09.225353 13216 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:09.225874 13216 heartbeater.cc:511] Master 127.11.158.190:39859 requested a full tablet report, sending...
I20260926 16:37:09.227484 13012 ts_manager.cc:194] Registered new tserver with Master: e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129:34989)
I20260926 16:37:09.228722 13012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:33895
I20260926 16:37:09.228739 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 13085
I20260926 16:37:09.228860 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:09.229118 11898 external_mini_cluster.cc:1428] Setting key 7db29c8dd5ffcff7740259895f7ec428
I20260926 16:37:09.235247 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.227466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:42943 (local address 127.11.158.190:39859)
0926 16:37:09.227579 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.227583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.227726 (+ 143us) server_negotiation.cc:407] Connection header received
0926 16:37:09.227860 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.227863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.227909 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.227991 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:09.228441 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.229177 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.230273 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.230564 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.230681 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.231809 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:09.231813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:09.231816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:09.231844 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:09.233688 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.234095 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.234100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.234102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.234183 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.234459 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.234464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.234466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.234632 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:09.234739 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.234983 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.235096 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:37:09.235255 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.227386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39859 (local address 127.11.158.129:42943)
0926 16:37:09.227657 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.227671 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.227764 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.228006 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.228009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.228206 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:09.228295 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.228300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.229333 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.229336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.230172 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.230182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.230685 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.230688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.230783 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.231283 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:09.231296 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:09.231653 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:09.233791 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.233796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.233798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.233994 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.234280 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.234283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.234285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.234368 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.234727 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:09.234730 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:09.234828 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.234933 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.235069 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:09.256886 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39859
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:09.367043 13225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:09.367293 13225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:09.367336 13225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:09.371313 13225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:09.371369 13225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:09.371443 13225 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:09.371526 13225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:09.376910 13225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:39859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:09.377960 13225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:09.378979 13225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:09.388747 13230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:09.388813 13231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:09.388808 13233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:09.389001 13225 server_base.cc:1034] running on GCE node
I20260926 16:37:09.389484 13225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:09.390179 13225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:09.391389 13225 hybrid_clock.cc:648] HybridClock initialized: now 1790440629391365 us; error 41 us; skew 500 ppm
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:09.394157 13225 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:09.395032 13225 webserver.cc:460] Webserver started at http://127.11.158.130:43943/ using document root <none> and password file <none>
I20260926 16:37:09.395550 13225 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:09.395635 13225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:09.395809 13225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:09.397887 13225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "d3232d5092cd410da5c61c1a5b9d6be9"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "9b44b94b61b8132278246a3064ec258c"
server_key_iv: "81e7b0df24582804efccf87f73a87f44"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.398375 13225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "d3232d5092cd410da5c61c1a5b9d6be9"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "9b44b94b61b8132278246a3064ec258c"
server_key_iv: "81e7b0df24582804efccf87f73a87f44"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.401799 13225 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 16:37:09.404008 13240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:09.405290 13225 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:09.405411 13225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "d3232d5092cd410da5c61c1a5b9d6be9"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "9b44b94b61b8132278246a3064ec258c"
server_key_iv: "81e7b0df24582804efccf87f73a87f44"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.405509 13225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:09.417884 13225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:09.420739 13225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:09.420917 13225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:09.422253 13225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:09.422323 13225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:09.422389 13225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:09.422436 13225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:09.433259 13225 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:44559
I20260926 16:37:09.433447 13355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:44559 every 8 connection(s)
I20260926 16:37:09.434327 13225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:09.435985 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.423142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:55679 (local address 127.11.158.190:39859)
0926 16:37:09.423266 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.423270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.424247 (+ 977us) server_negotiation.cc:407] Connection header received
0926 16:37:09.425465 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.425468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.425528 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.425605 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:09.426942 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.427630 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.428902 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.429247 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.429362 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.432198 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:09.432202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:09.432205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:09.432249 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:09.434038 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.434541 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.434545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.434547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.434645 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.435040 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.435044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.435046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.435258 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:09.435353 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.435751 (+ 398us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.435843 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:09.436808 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.423627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39859 (local address 127.11.158.130:55679)
0926 16:37:09.424095 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.424136 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.425187 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.425771 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.425778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.426084 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:09.426777 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.426801 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.427752 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.427756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.428776 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.428785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.429413 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.429417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.429585 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.430135 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:09.430151 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:09.432042 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:09.434138 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.434142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.434153 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.434397 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.434752 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.434755 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.434757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.434911 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.435348 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:09.435352 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:09.435554 (+ 202us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.435704 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.435931 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":131,"threads_started":1}
I20260926 16:37:09.438059 13356 heartbeater.cc:347] Connected to a master server at 127.11.158.190:39859
I20260926 16:37:09.438190 13356 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:09.438721 13356 heartbeater.cc:511] Master 127.11.158.190:39859 requested a full tablet report, sending...
I20260926 16:37:09.439747 13012 ts_manager.cc:194] Registered new tserver with Master: d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130:44559)
I20260926 16:37:09.440503 13012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:55679
I20260926 16:37:09.443037 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 13225
I20260926 16:37:09.443171 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:09.443454 11898 external_mini_cluster.cc:1428] Setting key b16e93614b923908520e401a4ec60fa6
I20260926 16:37:09.448527 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.440161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39859 (local address 127.11.158.130:60081)
0926 16:37:09.440449 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.440464 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.440591 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.440875 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.440878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.441097 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:09.441197 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.441203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.442182 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.442184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.442731 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.442738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.443418 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.443421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.443535 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.444250 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:09.444264 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:09.444768 (+ 504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:09.446819 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.446824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.446827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.447127 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.447392 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.447396 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.447399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.447513 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.447903 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:09.447906 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:09.448013 (+ 107us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.448136 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.448282 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":107,"threads_started":1}
I20260926 16:37:09.448591 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.440223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:60081 (local address 127.11.158.190:39859)
0926 16:37:09.440359 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.440364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.440546 (+ 182us) server_negotiation.cc:407] Connection header received
0926 16:37:09.440694 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.440699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.440769 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.440846 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:09.441326 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.442064 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.442857 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.443182 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.443299 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.444917 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:09.444921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:09.444923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:09.444946 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:09.446682 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.447226 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.447229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.447230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.447300 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.447614 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.447616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.447617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.447798 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:09.447885 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.448214 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.448352 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:09.471280 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39859
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:09.580157 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:09.580444 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:09.580488 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:09.584749 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:09.584846 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:09.584919 13365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:09.584977 13365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:09.590507 13365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:39859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:09.591513 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:09.592556 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:09.602068 13373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:09.602135 13370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:09.602193 13365 server_base.cc:1034] running on GCE node
W20260926 16:37:09.602068 13371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:09.602782 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:09.603345 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:09.604512 13365 hybrid_clock.cc:648] HybridClock initialized: now 1790440629604469 us; error 28 us; skew 500 ppm
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:09.607271 13365 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:09.608188 13365 webserver.cc:460] Webserver started at http://127.11.158.131:39035/ using document root <none> and password file <none>
I20260926 16:37:09.608801 13365 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:09.608892 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:09.609130 13365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:09.611176 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "fb4a90ee20b34e258b359fc021ad2770"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "a788273e6bc9008858bb1653cb1539b0"
server_key_iv: "09cc457fc005879ddb4772bc1faff0c0"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.611734 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "fb4a90ee20b34e258b359fc021ad2770"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "a788273e6bc9008858bb1653cb1539b0"
server_key_iv: "09cc457fc005879ddb4772bc1faff0c0"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.615443 13365 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:37:09.617848 13380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:09.619081 13365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:09.619225 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "fb4a90ee20b34e258b359fc021ad2770"
format_stamp: "Formatted at 2026-09-26 16:37:09 on dist-test-slave-p39x"
server_key: "a788273e6bc9008858bb1653cb1539b0"
server_key_iv: "09cc457fc005879ddb4772bc1faff0c0"
server_key_version: "encryptionkey@0"
I20260926 16:37:09.619339 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:09.638765 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:09.641939 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:09.642155 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:09.643643 13365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:09.643714 13365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:09.643782 13365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:09.643831 13365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:09.654937 13365 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:41509
I20260926 16:37:09.655002 13495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:41509 every 8 connection(s)
I20260926 16:37:09.656031 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:09.657440 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 13365
I20260926 16:37:09.657567 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:09.657863 11898 external_mini_cluster.cc:1428] Setting key 8da20d1441e32aa272913c79e13f139a
I20260926 16:37:09.658209 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.644504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:59149 (local address 127.11.158.190:39859)
0926 16:37:09.644636 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.644642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.645728 (+ 1086us) server_negotiation.cc:407] Connection header received
0926 16:37:09.646837 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.646844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.646916 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.647016 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:09.648821 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.649424 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.650570 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.650862 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.650975 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.654201 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:09.654203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:09.654205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:09.654228 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:09.655888 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.656480 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.656485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.656488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.656584 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.656967 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.656970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.656974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.657225 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:09.657306 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.657960 (+ 654us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.658074 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:37:09.659180 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.645069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39859 (local address 127.11.158.131:59149)
0926 16:37:09.645586 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.645627 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.646555 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.647355 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.647366 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.647860 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:09.648636 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.648662 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.649554 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.649559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.650436 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.650444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.651037 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.651044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.651284 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.651937 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:09.651955 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:09.654043 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:09.656008 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.656012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.656035 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.656331 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.656687 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.656690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.656692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.656850 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.657333 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:09.657341 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:09.657678 (+ 337us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.657879 (+ 201us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.658192 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260926 16:37:09.660425 13496 heartbeater.cc:347] Connected to a master server at 127.11.158.190:39859
I20260926 16:37:09.660602 13496 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:09.661158 13496 heartbeater.cc:511] Master 127.11.158.190:39859 requested a full tablet report, sending...
I20260926 16:37:09.662181 13013 ts_manager.cc:194] Registered new tserver with Master: fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131:41509)
I20260926 16:37:09.662801 13013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:59149
I20260926 16:37:09.669219 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.662264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39859 (local address 127.11.158.131:50249)
0926 16:37:09.663164 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.663176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.663281 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.663529 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.663532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.663741 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:09.663856 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.663862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.664646 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.664650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.665254 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.665262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.665668 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.665670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.665734 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.666135 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:09.666145 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:09.666404 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:09.668026 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.668028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.668030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.668200 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.668471 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:09.668473 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:09.668474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:09.668551 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:09.668854 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:09.668856 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:09.668921 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.668997 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.669101 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":95,"threads_started":1}
I20260926 16:37:09.669229 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.662354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:50249 (local address 127.11.158.190:39859)
0926 16:37:09.662461 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.662465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.663243 (+ 778us) server_negotiation.cc:407] Connection header received
0926 16:37:09.663390 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.663392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.663428 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.663491 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:09.663960 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.664508 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.665356 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.665575 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.665651 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.666516 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:09.666537 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:09.666539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:09.666558 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:09.667918 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.668279 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.668281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.668282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.668395 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.668627 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.668630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.668631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.668778 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:09.668850 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.669054 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.669132 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:37:09.671712 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:37:09 dist-test-slave-p39x krb5kdc[12968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:09.696270 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.687797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33840 (local address 127.11.158.190:39859)
0926 16:37:09.687921 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.687924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.687937 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:09.687977 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.687980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.688019 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.688095 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:09.689000 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.689535 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.690352 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.690580 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.690677 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.693389 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:09.693393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:09.693396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:09.693425 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:09.694763 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.695161 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.695164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.695165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.695248 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:09.695490 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:09.695492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:09.695493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:09.695683 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:09.695787 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.695995 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.696141 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:09.699286 13013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:09.702785 13013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:09.718756 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.713380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.11.158.131:41509)
0926 16:37:09.714009 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.714023 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.714054 (+ 31us) server_negotiation.cc:407] Connection header received
0926 16:37:09.714111 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.714138 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.714317 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.714712 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:09.714828 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.715911 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.717081 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.717882 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.718060 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.718202 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.718497 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.718597 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":102,"threads_started":1}
I20260926 16:37:09.718794 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.712561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:41509 (local address 127.0.0.1:41782)
0926 16:37:09.713688 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.713700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.713766 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.714469 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.714472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.714489 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:09.714579 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.714585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.716034 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.716038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.716945 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.716954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.718040 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.718044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.718240 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.718330 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.718449 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.718621 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":96,"thread_start_us":119,"threads_started":1}
I20260926 16:37:09.719208 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.712611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47644 (local address 127.11.158.130:44559)
0926 16:37:09.713071 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.713090 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.713357 (+ 267us) server_negotiation.cc:407] Connection header received
0926 16:37:09.713444 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.713449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.713699 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.713870 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:09.714184 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.715340 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.717199 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.717944 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.718075 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.718202 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.718958 (+ 756us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.719069 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":93,"threads_started":1}
I20260926 16:37:09.718794 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.712257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44559 (local address 127.0.0.1:47644)
0926 16:37:09.712999 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.713064 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.713220 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.713884 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.713890 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.713915 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:09.714044 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.714050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.715509 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.715512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.717072 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.717083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.718049 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.718051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.718238 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.718330 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.718449 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.718621 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":360,"thread_start_us":120,"threads_started":1}
I20260926 16:37:09.719854 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.712266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34989 (local address 127.0.0.1:36814)
0926 16:37:09.713457 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.713469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.713541 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.714216 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.714219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.714235 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:09.714325 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.714331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.715419 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.715425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.717072 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.717083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.719373 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.719378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.719471 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.719495 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.719556 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.719623 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1132,"mutex_wait_us":107,"thread_start_us":629,"threads_started":1}
I20260926 16:37:09.720212 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.713008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.11.158.129:34989)
0926 16:37:09.713778 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.713819 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.713846 (+ 27us) server_negotiation.cc:407] Connection header received
0926 16:37:09.713915 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.713920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.714101 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.714250 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:09.714588 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.715305 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.718265 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.719234 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.719810 (+ 576us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.719936 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.719986 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.720071 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":81,"threads_started":1}
I20260926 16:37:09.722486 13424 tablet_service.cc:1467] Processing CreateTablet for tablet bf5e9c0a4b56434db04a769c0254e324 (DEFAULT_TABLE table=test-table [id=78b3d65af48e41c590e6b7c87774bcd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:09.722587 13290 tablet_service.cc:1467] Processing CreateTablet for tablet bf5e9c0a4b56434db04a769c0254e324 (DEFAULT_TABLE table=test-table [id=78b3d65af48e41c590e6b7c87774bcd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:09.722785 13150 tablet_service.cc:1467] Processing CreateTablet for tablet bf5e9c0a4b56434db04a769c0254e324 (DEFAULT_TABLE table=test-table [id=78b3d65af48e41c590e6b7c87774bcd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:09.723686 13150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5e9c0a4b56434db04a769c0254e324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:09.723693 13424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5e9c0a4b56434db04a769c0254e324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:09.723688 13290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5e9c0a4b56434db04a769c0254e324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:09.730482 13519 tablet_bootstrap.cc:492] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Bootstrap starting.
I20260926 16:37:09.731688 13520 tablet_bootstrap.cc:492] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Bootstrap starting.
I20260926 16:37:09.731956 13521 tablet_bootstrap.cc:492] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Bootstrap starting.
I20260926 16:37:09.733150 13519 tablet_bootstrap.cc:654] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:09.733911 13521 tablet_bootstrap.cc:654] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:09.734025 13520 tablet_bootstrap.cc:654] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:09.734028 13519 log.cc:824] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:09.734695 13521 log.cc:824] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:09.734834 13520 log.cc:824] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:09.736073 13519 tablet_bootstrap.cc:492] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: No bootstrap required, opened a new log
I20260926 16:37:09.736277 13519 ts_tablet_manager.cc:1397] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:09.736557 13520 tablet_bootstrap.cc:492] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: No bootstrap required, opened a new log
I20260926 16:37:09.736577 13521 tablet_bootstrap.cc:492] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: No bootstrap required, opened a new log
I20260926 16:37:09.736748 13521 ts_tablet_manager.cc:1397] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:37:09.736753 13520 ts_tablet_manager.cc:1397] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:37:09.739785 13520 raft_consensus.cc:348] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.740022 13520 raft_consensus.cc:374] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:09.740084 13520 raft_consensus.cc:729] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3732e84eab84b8f9f75c33fa0eb17be, State: Initialized, Role: FOLLOWER
I20260926 16:37:09.740403 13519 raft_consensus.cc:348] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.740490 13521 raft_consensus.cc:348] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.740655 13519 raft_consensus.cc:374] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:09.740589 13520 consensus_queue.cc:260] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.740688 13521 raft_consensus.cc:374] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:09.740715 13519 raft_consensus.cc:729] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3232d5092cd410da5c61c1a5b9d6be9, State: Initialized, Role: FOLLOWER
I20260926 16:37:09.740736 13521 raft_consensus.cc:729] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4a90ee20b34e258b359fc021ad2770, State: Initialized, Role: FOLLOWER
I20260926 16:37:09.741160 13521 consensus_queue.cc:260] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.741160 13519 consensus_queue.cc:260] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.741591 13520 ts_tablet_manager.cc:1428] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:37:09.741876 13216 heartbeater.cc:503] Master 127.11.158.190:39859 was elected leader, sending a full tablet report...
I20260926 16:37:09.742084 13519 ts_tablet_manager.cc:1428] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:37:09.742403 13356 heartbeater.cc:503] Master 127.11.158.190:39859 was elected leader, sending a full tablet report...
I20260926 16:37:09.743160 13521 ts_tablet_manager.cc:1428] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:37:09.743499 13496 heartbeater.cc:503] Master 127.11.158.190:39859 was elected leader, sending a full tablet report...
W20260926 16:37:09.907058 13497 tablet.cc:2369] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:09.935706 13357 tablet.cc:2369] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:09.972049 13217 tablet.cc:2369] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:09.985828 13527 raft_consensus.cc:482] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:09.986076 13527 raft_consensus.cc:504] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.987206 13527 leader_election.cc:288] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130:44559), e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129:34989)
I20260926 16:37:09.990773 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.987542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:45097 (local address 127.11.158.130:44559)
0926 16:37:09.987737 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.987741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.987758 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:37:09.987878 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.987881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.987920 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.987999 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:09.988274 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.988847 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.989868 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.990252 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.990375 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.990428 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.990596 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.990664 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260926 16:37:09.990772 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.987480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44559 (local address 127.11.158.131:45097)
0926 16:37:09.987639 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.987657 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.987755 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.988021 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.988023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.988039 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:09.988164 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.988169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.988954 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.988960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.989734 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.989742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.990365 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.990368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.990468 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.990494 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.990578 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.990645 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 16:37:09.991374 13310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5e9c0a4b56434db04a769c0254e324" candidate_uuid: "fb4a90ee20b34e258b359fc021ad2770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" is_pre_election: true
I20260926 16:37:09.991665 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.987543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:47491 (local address 127.11.158.129:34989)
0926 16:37:09.987713 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:09.987718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:09.988074 (+ 356us) server_negotiation.cc:407] Connection header received
0926 16:37:09.988229 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:09.988232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:09.988282 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:09.988392 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:09.988992 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.989492 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.990621 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.991132 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.991266 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.991315 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:09.991489 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 16:37:09.991564 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 16:37:09.991681 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:09.987482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34989 (local address 127.11.158.131:47491)
0926 16:37:09.987977 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:09.987988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:09.988116 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:09.988409 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:09.988412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:09.988427 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:09.988880 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.988886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.989606 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.989613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.990491 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:09.990500 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:09.991264 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:09.991268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:09.991374 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:09.991390 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 16:37:09.991475 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:37:09.991538 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":185,"thread_start_us":148,"threads_started":1}
I20260926 16:37:09.991671 13310 raft_consensus.cc:2454] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb4a90ee20b34e258b359fc021ad2770 in term 0.
I20260926 16:37:09.992189 13384 leader_election.cc:302] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3232d5092cd410da5c61c1a5b9d6be9, fb4a90ee20b34e258b359fc021ad2770; no voters:
I20260926 16:37:09.992449 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5e9c0a4b56434db04a769c0254e324" candidate_uuid: "fb4a90ee20b34e258b359fc021ad2770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" is_pre_election: true
I20260926 16:37:09.992501 13527 raft_consensus.cc:2781] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:09.992601 13527 raft_consensus.cc:482] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:09.992659 13527 raft_consensus.cc:3037] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:09.992741 13170 raft_consensus.cc:2454] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb4a90ee20b34e258b359fc021ad2770 in term 0.
I20260926 16:37:09.993705 13527 raft_consensus.cc:504] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.993983 13527 leader_election.cc:288] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [CANDIDATE]: Term 1 election: Requested vote from peers d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130:44559), e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129:34989)
I20260926 16:37:09.994321 13310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5e9c0a4b56434db04a769c0254e324" candidate_uuid: "fb4a90ee20b34e258b359fc021ad2770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3232d5092cd410da5c61c1a5b9d6be9"
I20260926 16:37:09.994320 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5e9c0a4b56434db04a769c0254e324" candidate_uuid: "fb4a90ee20b34e258b359fc021ad2770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3732e84eab84b8f9f75c33fa0eb17be"
I20260926 16:37:09.994438 13170 raft_consensus.cc:3037] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:09.994439 13310 raft_consensus.cc:3037] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:09.995642 13170 raft_consensus.cc:2454] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4a90ee20b34e258b359fc021ad2770 in term 1.
I20260926 16:37:09.995642 13310 raft_consensus.cc:2454] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4a90ee20b34e258b359fc021ad2770 in term 1.
I20260926 16:37:09.995919 13384 leader_election.cc:302] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3232d5092cd410da5c61c1a5b9d6be9, fb4a90ee20b34e258b359fc021ad2770; no voters:
I20260926 16:37:09.996096 13527 raft_consensus.cc:2781] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:09.996291 13527 raft_consensus.cc:686] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 1 LEADER]: Becoming Leader. State: Replica: fb4a90ee20b34e258b359fc021ad2770, State: Running, Role: LEADER
I20260926 16:37:09.996657 13527 consensus_queue.cc:237] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:09.999588 13012 catalog_manager.cc:5495] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131). New cstate: current_term: 1 leader_uuid: "fb4a90ee20b34e258b359fc021ad2770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } health_report { overall_health: HEALTHY } } }
I20260926 16:37:10.089105 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.083494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.11.158.131:41509)
0926 16:37:10.083777 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.083782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.083801 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:37:10.083893 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.083896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.083992 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.084126 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:10.084527 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.085508 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.087022 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.087373 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.087496 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.087945 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.088587 (+ 642us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.088734 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.088892 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.088959 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 16:37:10.096414 13310 raft_consensus.cc:1261] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Refusing update from remote peer fb4a90ee20b34e258b359fc021ad2770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:10.096467 13170 raft_consensus.cc:1261] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Refusing update from remote peer fb4a90ee20b34e258b359fc021ad2770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:10.097250 13529 consensus_queue.cc:1035] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:10.097445 13527 consensus_queue.cc:1035] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:10.116132 13012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:10.116307 13012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:10.118644 13012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: Sending DeleteTablet for 3 replicas of tablet bf5e9c0a4b56434db04a769c0254e324
I20260926 16:37:10.119413 13290 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5e9c0a4b56434db04a769c0254e324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:10 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:47644
I20260926 16:37:10.119652 13424 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5e9c0a4b56434db04a769c0254e324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:10 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41782
I20260926 16:37:10.119863 13150 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5e9c0a4b56434db04a769c0254e324 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:10 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260926 16:37:10.120131 13543 tablet_replica.cc:330] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: stopping tablet replica
I20260926 16:37:10.120317 13542 tablet_replica.cc:330] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: stopping tablet replica
I20260926 16:37:10.120456 13543 raft_consensus.cc:2229] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:10.120354 13012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 16:37:10.120622 13542 raft_consensus.cc:2229] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:10.120798 13543 raft_consensus.cc:2258] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:10.120918 13542 raft_consensus.cc:2258] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:10.121137 13544 tablet_replica.cc:330] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: stopping tablet replica
W20260926 16:37:10.121130 13012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:10.121423 13544 raft_consensus.cc:2229] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:10.121745 13544 raft_consensus.cc:2258] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:10.123085 13543 ts_tablet_manager.cc:1905] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:10.123492 13542 ts_tablet_manager.cc:1905] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:10.125476 13544 ts_tablet_manager.cc:1905] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:10.125932 13543 ts_tablet_manager.cc:1918] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:10.126019 13543 log.cc:1196] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/bf5e9c0a4b56434db04a769c0254e324
I20260926 16:37:10.126339 13543 ts_tablet_manager.cc:1939] T bf5e9c0a4b56434db04a769c0254e324 P fb4a90ee20b34e258b359fc021ad2770: Deleting consensus metadata
I20260926 16:37:10.126632 13424 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3c088cb9714d5e88854171fb888a1c (DEFAULT_TABLE table=test-table [id=3dc3fcc99eb94200bb32312ab3ac1802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:10.126632 13290 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3c088cb9714d5e88854171fb888a1c (DEFAULT_TABLE table=test-table [id=3dc3fcc99eb94200bb32312ab3ac1802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:10.126853 13290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3c088cb9714d5e88854171fb888a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:10.126853 13424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3c088cb9714d5e88854171fb888a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:10.128080 13544 ts_tablet_manager.cc:1918] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:10.128166 13544 log.cc:1196] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/bf5e9c0a4b56434db04a769c0254e324
I20260926 16:37:10.128496 13544 ts_tablet_manager.cc:1939] T bf5e9c0a4b56434db04a769c0254e324 P e3732e84eab84b8f9f75c33fa0eb17be: Deleting consensus metadata
I20260926 16:37:10.128981 13521 tablet_bootstrap.cc:492] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: Bootstrap starting.
I20260926 16:37:10.128981 13519 tablet_bootstrap.cc:492] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: Bootstrap starting.
I20260926 16:37:10.129890 13519 tablet_bootstrap.cc:654] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:10.129890 13521 tablet_bootstrap.cc:654] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:10.130518 13542 ts_tablet_manager.cc:1918] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:10.130599 13542 log.cc:1196] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/bf5e9c0a4b56434db04a769c0254e324
I20260926 16:37:10.131369 13542 ts_tablet_manager.cc:1939] T bf5e9c0a4b56434db04a769c0254e324 P d3232d5092cd410da5c61c1a5b9d6be9: Deleting consensus metadata
I20260926 16:37:10.131366 12998 catalog_manager.cc:4841] TS fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131:41509): tablet bf5e9c0a4b56434db04a769c0254e324 (table test-table [id=78b3d65af48e41c590e6b7c87774bcd7]) successfully deleted
I20260926 16:37:10.131584 12999 catalog_manager.cc:4841] TS e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129:34989): tablet bf5e9c0a4b56434db04a769c0254e324 (table test-table [id=78b3d65af48e41c590e6b7c87774bcd7]) successfully deleted
I20260926 16:37:10.131955 13150 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3c088cb9714d5e88854171fb888a1c (DEFAULT_TABLE table=test-table [id=3dc3fcc99eb94200bb32312ab3ac1802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:10.132246 13150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3c088cb9714d5e88854171fb888a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:10.132503 13000 catalog_manager.cc:4841] TS d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130:44559): tablet bf5e9c0a4b56434db04a769c0254e324 (table test-table [id=78b3d65af48e41c590e6b7c87774bcd7]) successfully deleted
I20260926 16:37:10.133040 13519 tablet_bootstrap.cc:492] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: No bootstrap required, opened a new log
I20260926 16:37:10.133046 13521 tablet_bootstrap.cc:492] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: No bootstrap required, opened a new log
I20260926 16:37:10.133117 13521 ts_tablet_manager.cc:1397] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 16:37:10.133118 13519 ts_tablet_manager.cc:1397] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 16:37:10.133517 13519 raft_consensus.cc:348] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.133615 13519 raft_consensus.cc:374] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:10.133549 13521 raft_consensus.cc:348] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.133644 13519 raft_consensus.cc:729] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3232d5092cd410da5c61c1a5b9d6be9, State: Initialized, Role: FOLLOWER
I20260926 16:37:10.133663 13521 raft_consensus.cc:374] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:10.133698 13521 raft_consensus.cc:729] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4a90ee20b34e258b359fc021ad2770, State: Initialized, Role: FOLLOWER
I20260926 16:37:10.133757 13519 consensus_queue.cc:260] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.133827 13521 consensus_queue.cc:260] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.133980 13519 ts_tablet_manager.cc:1428] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:10.134056 13521 ts_tablet_manager.cc:1428] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:37:10.134711 13520 tablet_bootstrap.cc:492] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: Bootstrap starting.
I20260926 16:37:10.135667 13520 tablet_bootstrap.cc:654] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:10.136875 13520 tablet_bootstrap.cc:492] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: No bootstrap required, opened a new log
I20260926 16:37:10.136974 13520 ts_tablet_manager.cc:1397] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:10.137444 13520 raft_consensus.cc:348] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.137554 13520 raft_consensus.cc:374] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:10.137602 13520 raft_consensus.cc:729] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3732e84eab84b8f9f75c33fa0eb17be, State: Initialized, Role: FOLLOWER
I20260926 16:37:10.137727 13520 consensus_queue.cc:260] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.137992 13520 ts_tablet_manager.cc:1428] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:10.146090 13525 raft_consensus.cc:482] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:10.146232 13525 raft_consensus.cc:504] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.147403 13525 leader_election.cc:288] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130:44559), fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131:41509)
I20260926 16:37:10.152310 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.148113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:52337 (local address 127.11.158.130:44559)
0926 16:37:10.148285 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.148291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.148307 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:10.148382 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.148385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.148434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.148518 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:10.149098 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.149775 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.151025 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.151674 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.151820 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.151881 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.152082 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.152158 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 16:37:10.152554 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.147775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:42611 (local address 127.11.158.131:41509)
0926 16:37:10.148006 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.148014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.148118 (+ 104us) server_negotiation.cc:407] Connection header received
0926 16:37:10.148357 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.148362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.148421 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.148518 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:10.148861 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.149654 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.151230 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.151888 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.152027 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.152087 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.152325 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.152425 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260926 16:37:10.152606 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.147679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:41509 (local address 127.11.158.129:42611)
0926 16:37:10.148036 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.148050 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.148201 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.148543 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.148546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.148565 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:10.148729 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.148737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.149794 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.149797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.151097 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.151107 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.152008 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.152013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.152134 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.152163 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.152305 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.152411 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":31,"thread_start_us":108,"threads_started":1}
I20260926 16:37:10.152912 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.147586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44559 (local address 127.11.158.129:52337)
0926 16:37:10.147957 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.147974 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.148095 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.148693 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.148697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.148721 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:10.148919 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.148931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.149913 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.149916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.150860 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.150869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.151823 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.151827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.151927 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.151954 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.152509 (+ 555us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.152586 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260926 16:37:10.153376 13450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c088cb9714d5e88854171fb888a1c" candidate_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb4a90ee20b34e258b359fc021ad2770" is_pre_election: true
I20260926 16:37:10.153605 13450 raft_consensus.cc:2454] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3732e84eab84b8f9f75c33fa0eb17be in term 0.
I20260926 16:37:10.154053 13310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c088cb9714d5e88854171fb888a1c" candidate_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" is_pre_election: true
I20260926 16:37:10.154228 13310 raft_consensus.cc:2454] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3732e84eab84b8f9f75c33fa0eb17be in term 0.
I20260926 16:37:10.154613 13104 leader_election.cc:302] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3232d5092cd410da5c61c1a5b9d6be9, e3732e84eab84b8f9f75c33fa0eb17be; no voters:
I20260926 16:37:10.154879 13525 raft_consensus.cc:2781] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:10.154975 13525 raft_consensus.cc:482] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:10.155038 13525 raft_consensus.cc:3037] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:10.156090 13525 raft_consensus.cc:504] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.156605 13525 leader_election.cc:288] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [CANDIDATE]: Term 1 election: Requested vote from peers d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130:44559), fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131:41509)
I20260926 16:37:10.156934 13310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c088cb9714d5e88854171fb888a1c" candidate_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3232d5092cd410da5c61c1a5b9d6be9"
I20260926 16:37:10.156991 13450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c088cb9714d5e88854171fb888a1c" candidate_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb4a90ee20b34e258b359fc021ad2770"
I20260926 16:37:10.157097 13450 raft_consensus.cc:3037] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:10.157094 13310 raft_consensus.cc:3037] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:10.158160 13310 raft_consensus.cc:2454] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3732e84eab84b8f9f75c33fa0eb17be in term 1.
I20260926 16:37:10.158305 13450 raft_consensus.cc:2454] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3732e84eab84b8f9f75c33fa0eb17be in term 1.
I20260926 16:37:10.158496 13104 leader_election.cc:302] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3232d5092cd410da5c61c1a5b9d6be9, e3732e84eab84b8f9f75c33fa0eb17be; no voters:
I20260926 16:37:10.158682 13525 raft_consensus.cc:2781] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:10.158871 13525 raft_consensus.cc:686] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 1 LEADER]: Becoming Leader. State: Replica: e3732e84eab84b8f9f75c33fa0eb17be, State: Running, Role: LEADER
I20260926 16:37:10.159137 13525 consensus_queue.cc:237] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.161922 13012 catalog_manager.cc:5495] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be reported cstate change: term changed from 0 to 1, leader changed from <none> to e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129). New cstate: current_term: 1 leader_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } health_report { overall_health: UNKNOWN } } }
Sep 26 16:37:10 dist-test-slave-p39x krb5kdc[12968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:10.199558 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.190298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.11.158.190:39859)
0926 16:37:10.190434 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.190438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.190698 (+ 260us) server_negotiation.cc:407] Connection header received
0926 16:37:10.190830 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.190832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.190873 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.190941 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:10.191759 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.192280 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.193408 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.193621 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.193702 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.196284 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:10.196287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:10.196289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:10.196309 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:10.197780 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:10.198256 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:10.198262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:10.198264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:10.198339 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:10.198605 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:10.198609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:10.198611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:10.198779 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:10.198898 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.199251 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.199408 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":56}
I20260926 16:37:10.199599 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.190204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39859 (local address 127.0.0.1:33852)
0926 16:37:10.190597 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.190611 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.190736 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.190996 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.190999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.191527 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:10.191641 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.191648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.192408 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.192412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.193287 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.193297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.193734 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.193737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.193833 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.194436 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:10.194452 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:10.196149 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:10.197903 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:10.197914 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:10.197921 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:10.198158 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:10.198430 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:10.198433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:10.198435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:10.198516 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:10.198918 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:10.198926 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:10.199054 (+ 128us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.199181 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.199351 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":120,"threads_started":1}
I20260926 16:37:10.204006 13012 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:33852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 16:37:10.204865 13012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:10.209967 13290 tablet_service.cc:1467] Processing CreateTablet for tablet 772af7fa54b2423a98e794555249fe82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94e4691ab85142458473abc6a7b5e51d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:10.209967 13150 tablet_service.cc:1467] Processing CreateTablet for tablet 772af7fa54b2423a98e794555249fe82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94e4691ab85142458473abc6a7b5e51d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:10.210086 13424 tablet_service.cc:1467] Processing CreateTablet for tablet 772af7fa54b2423a98e794555249fe82 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94e4691ab85142458473abc6a7b5e51d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:10.210359 13290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772af7fa54b2423a98e794555249fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:10.210359 13150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772af7fa54b2423a98e794555249fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:10.210444 13424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772af7fa54b2423a98e794555249fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:10.213198 13521 tablet_bootstrap.cc:492] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: Bootstrap starting.
I20260926 16:37:10.213277 13520 tablet_bootstrap.cc:492] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: Bootstrap starting.
I20260926 16:37:10.214103 13520 tablet_bootstrap.cc:654] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:10.214152 13521 tablet_bootstrap.cc:654] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:10.214768 13519 tablet_bootstrap.cc:492] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: Bootstrap starting.
I20260926 16:37:10.215389 13520 tablet_bootstrap.cc:492] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: No bootstrap required, opened a new log
I20260926 16:37:10.215493 13520 ts_tablet_manager.cc:1397] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:10.215758 13519 tablet_bootstrap.cc:654] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:10.215939 13520 raft_consensus.cc:348] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.216056 13520 raft_consensus.cc:374] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:10.216097 13520 raft_consensus.cc:729] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3732e84eab84b8f9f75c33fa0eb17be, State: Initialized, Role: FOLLOWER
I20260926 16:37:10.216246 13520 consensus_queue.cc:260] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.216591 13520 ts_tablet_manager.cc:1428] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:37:10.216807 13525 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.216923 13525 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:10.217069 13519 tablet_bootstrap.cc:492] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: No bootstrap required, opened a new log
I20260926 16:37:10.217139 13519 ts_tablet_manager.cc:1397] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:10.217553 13519 raft_consensus.cc:348] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.217664 13519 raft_consensus.cc:374] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:10.217697 13519 raft_consensus.cc:729] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3232d5092cd410da5c61c1a5b9d6be9, State: Initialized, Role: FOLLOWER
I20260926 16:37:10.217815 13519 consensus_queue.cc:260] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.218044 13519 ts_tablet_manager.cc:1428] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:10.218039 13526 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.218240 13526 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:10.219394 13521 tablet_bootstrap.cc:492] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: No bootstrap required, opened a new log
I20260926 16:37:10.219485 13521 ts_tablet_manager.cc:1397] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260926 16:37:10.219870 13521 raft_consensus.cc:348] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.219985 13521 raft_consensus.cc:374] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:10.220014 13521 raft_consensus.cc:729] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4a90ee20b34e258b359fc021ad2770, State: Initialized, Role: FOLLOWER
I20260926 16:37:10.220130 13521 consensus_queue.cc:260] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.220409 13521 ts_tablet_manager.cc:1428] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:10.220423 13537 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.220563 13537 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:10.342443 13526 raft_consensus.cc:482] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:10.342595 13526 raft_consensus.cc:504] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.343798 13526 leader_election.cc:288] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129:34989), fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131:41509)
I20260926 16:37:10.349555 13518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.344258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:47885 (local address 127.11.158.131:41509)
0926 16:37:10.344433 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.344439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.344797 (+ 358us) server_negotiation.cc:407] Connection header received
0926 16:37:10.344917 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.344920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.344970 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.345057 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:10.345383 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.346172 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.348105 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.348724 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.348848 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.348899 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.349353 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.349427 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:37:10.349555 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.344196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:38279 (local address 127.11.158.129:34989)
0926 16:37:10.344391 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.344396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.344672 (+ 276us) server_negotiation.cc:407] Connection header received
0926 16:37:10.344843 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.344850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.344909 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.344991 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:10.345383 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.346172 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.348104 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.348727 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.348848 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.348899 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.349354 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.349420 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 16:37:10.349696 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.344072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34989 (local address 127.11.158.130:38279)
0926 16:37:10.344562 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.344578 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.344708 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.345007 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.345026 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.345047 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:10.345266 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.345273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.346320 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.346324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.347957 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.347968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.348892 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.348897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.349158 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.349231 (+ 73us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.349330 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.349471 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":378,"thread_start_us":161,"threads_started":1}
I20260926 16:37:10.349892 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.344090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:41509 (local address 127.11.158.130:47885)
0926 16:37:10.344696 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.344708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.344797 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.345055 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.345057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.345090 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:10.345266 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.345273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.346320 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.346324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.347960 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.347968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.348892 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.348897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.349133 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.349232 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.349330 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.349468 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":314,"spinlock_wait_cycles":26496,"thread_start_us":136,"threads_started":1}
I20260926 16:37:10.350234 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772af7fa54b2423a98e794555249fe82" candidate_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" is_pre_election: true
I20260926 16:37:10.350422 13170 raft_consensus.cc:2454] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3232d5092cd410da5c61c1a5b9d6be9 in term 0.
I20260926 16:37:10.350463 13450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772af7fa54b2423a98e794555249fe82" candidate_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb4a90ee20b34e258b359fc021ad2770" is_pre_election: true
I20260926 16:37:10.350644 13450 raft_consensus.cc:2454] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3232d5092cd410da5c61c1a5b9d6be9 in term 0.
I20260926 16:37:10.350826 13243 leader_election.cc:302] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3232d5092cd410da5c61c1a5b9d6be9, e3732e84eab84b8f9f75c33fa0eb17be; no voters:
I20260926 16:37:10.351146 13526 raft_consensus.cc:2781] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:10.351212 13526 raft_consensus.cc:482] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:10.351243 13526 raft_consensus.cc:3037] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:10.352312 13526 raft_consensus.cc:504] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.352788 13526 leader_election.cc:288] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [CANDIDATE]: Term 1 election: Requested vote from peers e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129:34989), fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131:41509)
I20260926 16:37:10.353068 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772af7fa54b2423a98e794555249fe82" candidate_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3732e84eab84b8f9f75c33fa0eb17be"
I20260926 16:37:10.353148 13450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772af7fa54b2423a98e794555249fe82" candidate_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb4a90ee20b34e258b359fc021ad2770"
I20260926 16:37:10.353214 13170 raft_consensus.cc:3037] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:10.353291 13450 raft_consensus.cc:3037] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:10.354230 13170 raft_consensus.cc:2454] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3232d5092cd410da5c61c1a5b9d6be9 in term 1.
I20260926 16:37:10.354306 13450 raft_consensus.cc:2454] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3232d5092cd410da5c61c1a5b9d6be9 in term 1.
I20260926 16:37:10.354504 13243 leader_election.cc:302] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3232d5092cd410da5c61c1a5b9d6be9, e3732e84eab84b8f9f75c33fa0eb17be; no voters:
I20260926 16:37:10.354713 13526 raft_consensus.cc:2781] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:10.354925 13526 raft_consensus.cc:686] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 LEADER]: Becoming Leader. State: Replica: d3232d5092cd410da5c61c1a5b9d6be9, State: Running, Role: LEADER
I20260926 16:37:10.355175 13526 consensus_queue.cc:237] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } }
I20260926 16:37:10.356266 13563 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: TxnStatusTablet state changed. Reason: New leader d3232d5092cd410da5c61c1a5b9d6be9. Latest consensus state: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.356405 13563 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:10.356839 13565 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 16:37:10.356995 13565 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 16:37:10.357468 13012 catalog_manager.cc:5495] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130). New cstate: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:10.395628 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.391198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.11.158.130:44559)
0926 16:37:10.391316 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.391320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.391334 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:37:10.391505 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.391507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.391561 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.391648 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:10.391936 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.392808 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.394078 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.394293 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.394377 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.394707 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.395239 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.395328 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.395442 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.395489 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:37:10.395659 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.391087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44559 (local address 127.0.0.1:47654)
0926 16:37:10.391238 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.391255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.391360 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.391684 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.391687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.391696 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:10.391818 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.391826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.392935 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.392939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.393938 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.393950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.394407 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.394411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.394505 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.394561 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.395329 (+ 768us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.395346 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.395425 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.395514 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 16:37:10.398653 13170 raft_consensus.cc:1261] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Refusing update from remote peer d3232d5092cd410da5c61c1a5b9d6be9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:10.398793 13450 raft_consensus.cc:1261] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Refusing update from remote peer d3232d5092cd410da5c61c1a5b9d6be9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:10.399219 13526 consensus_queue.cc:1035] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:10.399379 13563 consensus_queue.cc:1035] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:10.400857 13525 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: TxnStatusTablet state changed. Reason: New leader d3232d5092cd410da5c61c1a5b9d6be9. Latest consensus state: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.400974 13525 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:10.401876 13537 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: TxnStatusTablet state changed. Reason: New leader d3232d5092cd410da5c61c1a5b9d6be9. Latest consensus state: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.401993 13537 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:10.403828 13526 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.403961 13526 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:10.404179 13533 mvcc.cc:205] Tried to move back new op lower bound from 7333644822107484160 to 7333644821936947200. Current Snapshot: MvccSnapshot[applied={T|T < 7333644822107484160}]
I20260926 16:37:10.404631 13532 mvcc.cc:205] Tried to move back new op lower bound from 7333644822107484160 to 7333644821936947200. Current Snapshot: MvccSnapshot[applied={T|T < 7333644822107484160}]
I20260926 16:37:10.404794 13537 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.404882 13537 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P fb4a90ee20b34e258b359fc021ad2770: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:10.403841 13563 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.405342 13563 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P d3232d5092cd410da5c61c1a5b9d6be9: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:10.406245 13534 mvcc.cc:205] Tried to move back new op lower bound from 7333644822107484160 to 7333644821936947200. Current Snapshot: MvccSnapshot[applied={T|T < 7333644822107484160}]
I20260926 16:37:10.407131 13525 tablet_replica.cc:439] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3732e84eab84b8f9f75c33fa0eb17be" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34989 } } peers { permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 } } peers { permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 } } }
I20260926 16:37:10.407236 13525 tablet_replica.cc:442] T 772af7fa54b2423a98e794555249fe82 P e3732e84eab84b8f9f75c33fa0eb17be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:10.414318 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.410644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.11.158.129:34989)
0926 16:37:10.410945 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.410948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.410958 (+ 10us) server_negotiation.cc:407] Connection header received
0926 16:37:10.411029 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.411031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.411079 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.411194 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:10.411426 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.411925 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.412879 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.413105 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.413212 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.413486 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.413931 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.414063 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.414127 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.414182 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260926 16:37:10.425589 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.421414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44559 (local address 127.11.158.129:58779)
0926 16:37:10.421717 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.421729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.421809 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.422111 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.422114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.422137 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:10.422239 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.422244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.423315 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.423320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.424132 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.424142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.424800 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.424804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.424941 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.425004 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.425329 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.425335 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.425387 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.425451 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260926 16:37:10.425586 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.421599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:58779 (local address 127.11.158.130:44559)
0926 16:37:10.421744 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.421749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.421769 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:37:10.421905 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.421908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.421960 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.422062 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:10.422364 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.423159 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.424256 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.424632 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.424775 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.425159 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.425244 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.425317 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.425397 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.425452 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:37:10.432287 13450 raft_consensus.cc:1261] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Refusing update from remote peer e3732e84eab84b8f9f75c33fa0eb17be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:10.432430 13310 raft_consensus.cc:1261] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Refusing update from remote peer e3732e84eab84b8f9f75c33fa0eb17be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:10.432782 13525 consensus_queue.cc:1035] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4a90ee20b34e258b359fc021ad2770" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:10.432981 13550 consensus_queue.cc:1035] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3232d5092cd410da5c61c1a5b9d6be9" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:10.467768 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.456093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:42957 (local address 127.11.158.129:34989)
0926 16:37:10.456242 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.456248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.464677 (+ 8429us) server_negotiation.cc:407] Connection header received
0926 16:37:10.464799 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.464802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.464851 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.464937 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:10.465172 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.465675 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.466521 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.466825 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.466929 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.467290 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.467375 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.467450 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.467558 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.467614 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 16:37:10.467777 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.455430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34989 (local address 127.11.158.130:42957)
0926 16:37:10.464556 (+ 9126us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:10.464588 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:10.464713 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:10.464946 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:10.464950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:10.464957 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:10.465061 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.465068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.465781 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.465786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.466400 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:10.466409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.466960 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.466964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:10.467062 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.467112 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:10.467470 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:10.467477 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:37:10.467548 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:37:10.467627 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9016,"thread_start_us":95,"threads_started":1}
I20260926 16:37:10.493716 13012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:10.493907 13012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:10.496325 13012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6d92e49122146798e112c928dfbce33: Sending DeleteTablet for 3 replicas of tablet 3e3c088cb9714d5e88854171fb888a1c
I20260926 16:37:10.497152 13290 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3c088cb9714d5e88854171fb888a1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:10 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:47644
I20260926 16:37:10.497359 13542 tablet_replica.cc:330] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: stopping tablet replica
I20260926 16:37:10.497417 13424 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3c088cb9714d5e88854171fb888a1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:10 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41782
I20260926 16:37:10.497500 13542 raft_consensus.cc:2229] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:10.497586 13543 tablet_replica.cc:330] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: stopping tablet replica
I20260926 16:37:10.497644 13542 raft_consensus.cc:2258] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:10.497701 13543 raft_consensus.cc:2229] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:10.497810 13543 raft_consensus.cc:2258] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:10.498070 13150 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3c088cb9714d5e88854171fb888a1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:10 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260926 16:37:10.498263 13544 tablet_replica.cc:330] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: stopping tablet replica
I20260926 16:37:10.498371 13544 raft_consensus.cc:2229] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:10.498593 13542 ts_tablet_manager.cc:1905] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:10.498688 13544 raft_consensus.cc:2258] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:10.499567 13543 ts_tablet_manager.cc:1905] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:10.501573 13542 ts_tablet_manager.cc:1918] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:10.501642 13542 log.cc:1196] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/3e3c088cb9714d5e88854171fb888a1c
I20260926 16:37:10.501896 13542 ts_tablet_manager.cc:1939] T 3e3c088cb9714d5e88854171fb888a1c P d3232d5092cd410da5c61c1a5b9d6be9: Deleting consensus metadata
I20260926 16:37:10.502660 13000 catalog_manager.cc:4841] TS d3232d5092cd410da5c61c1a5b9d6be9 (127.11.158.130:44559): tablet 3e3c088cb9714d5e88854171fb888a1c (table test-table [id=3dc3fcc99eb94200bb32312ab3ac1802]) successfully deleted
I20260926 16:37:10.503546 13544 ts_tablet_manager.cc:1905] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:10.506457 13544 ts_tablet_manager.cc:1918] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:10.506552 13544 log.cc:1196] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/3e3c088cb9714d5e88854171fb888a1c
I20260926 16:37:10.506834 13544 ts_tablet_manager.cc:1939] T 3e3c088cb9714d5e88854171fb888a1c P e3732e84eab84b8f9f75c33fa0eb17be: Deleting consensus metadata
I20260926 16:37:10.507593 13543 ts_tablet_manager.cc:1918] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:10.507685 13543 log.cc:1196] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/3e3c088cb9714d5e88854171fb888a1c
I20260926 16:37:10.507648 12999 catalog_manager.cc:4841] TS e3732e84eab84b8f9f75c33fa0eb17be (127.11.158.129:34989): tablet 3e3c088cb9714d5e88854171fb888a1c (table test-table [id=3dc3fcc99eb94200bb32312ab3ac1802]) successfully deleted
I20260926 16:37:10.507942 13543 ts_tablet_manager.cc:1939] T 3e3c088cb9714d5e88854171fb888a1c P fb4a90ee20b34e258b359fc021ad2770: Deleting consensus metadata
I20260926 16:37:10.508708 12998 catalog_manager.cc:4841] TS fb4a90ee20b34e258b359fc021ad2770 (127.11.158.131:41509): tablet 3e3c088cb9714d5e88854171fb888a1c (table test-table [id=3dc3fcc99eb94200bb32312ab3ac1802]) successfully deleted
Sep 26 16:37:10 dist-test-slave-p39x krb5kdc[12968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440629, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:10.520974 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.508738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.11.158.129:34989)
0926 16:37:10.508853 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.508858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.509530 (+ 672us) server_negotiation.cc:407] Connection header received
0926 16:37:10.509669 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.509673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.509721 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.509803 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:10.510757 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.511537 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.512754 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.513103 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.513206 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.517011 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:10.517021 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:10.517046 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:10.517073 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:10.519074 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:10.519552 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:10.519558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:10.519560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:10.519663 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:10.519940 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:10.519943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:10.519945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:10.520241 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:10.520469 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.520682 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.520801 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 16:37:10.521540 13210 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36834
I20260926 16:37:10.532691 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:10.523377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33858 (local address 127.11.158.190:39859)
0926 16:37:10.523502 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:10.523506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:10.523730 (+ 224us) server_negotiation.cc:407] Connection header received
0926 16:37:10.523830 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:10.523833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:10.523892 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:10.523986 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:10.524945 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.525733 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.526904 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:10.527256 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:10.527360 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:10.528940 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:10.528944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:10.528946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:10.528973 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:10.530663 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:10.531155 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:10.531159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:10.531162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:10.531271 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:10.531566 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:10.531569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:10.531571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:10.531779 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:10.531905 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:10.532158 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 16:37:10.532280 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 16:37:10.533289 13012 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33858
I20260926 16:37:10.534706 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 13085
I20260926 16:37:10.542148 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 13225
I20260926 16:37:10.548805 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 13365
I20260926 16:37:10.555176 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 12981
2026-09-26T16:37:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3046 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:10 dist-test-slave-p39x krb5kdc[13587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:10 dist-test-slave-p39x krb5kdc[13587](info): set up 2 sockets
Sep 26 16:37:10 dist-test-slave-p39x krb5kdc[13587](info): commencing operation
krb5kdc: starting...
W20260926 16:37:11.593427 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:11 dist-test-slave-p39x krb5kdc[13587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:11.721978 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40583
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:40583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:11.835959 13600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:11.836326 13600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:11.836434 13600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:11.840615 13600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:11.840739 13600 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 16:37:11.840772 13600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:11.840808 13600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:11.840870 13600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:11.840907 13600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:11.846541 13600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:40583
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:11.847769 13600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:11.849036 13600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:11.858547 13608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:11.858562 13606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:11.858605 13605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:11.858764 13600 server_base.cc:1034] running on GCE node
I20260926 16:37:11.859581 13600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:11.860909 13600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:11.862130 13600 hybrid_clock.cc:648] HybridClock initialized: now 1790440631862112 us; error 35 us; skew 500 ppm
Sep 26 16:37:11 dist-test-slave-p39x krb5kdc[13587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440631, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:11.865216 13600 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:11.866144 13600 webserver.cc:460] Webserver started at http://127.11.158.190:44715/ using document root <none> and password file <none>
I20260926 16:37:11.866698 13600 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:11.866761 13600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:11.866935 13600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:11.869119 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "0532e93e69f64d81a517db3adb4a14c8"
format_stamp: "Formatted at 2026-09-26 16:37:11 on dist-test-slave-p39x"
server_key: "ffd652cf7d9a006f2babc26c681059fe"
server_key_iv: "6421ed4b37932fab0bbdcfae45ede2e2"
server_key_version: "encryptionkey@0"
I20260926 16:37:11.869663 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "0532e93e69f64d81a517db3adb4a14c8"
format_stamp: "Formatted at 2026-09-26 16:37:11 on dist-test-slave-p39x"
server_key: "ffd652cf7d9a006f2babc26c681059fe"
server_key_iv: "6421ed4b37932fab0bbdcfae45ede2e2"
server_key_version: "encryptionkey@0"
I20260926 16:37:11.873776 13600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:37:11.876629 13615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:11.878151 13600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:37:11.878309 13600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "0532e93e69f64d81a517db3adb4a14c8"
format_stamp: "Formatted at 2026-09-26 16:37:11 on dist-test-slave-p39x"
server_key: "ffd652cf7d9a006f2babc26c681059fe"
server_key_iv: "6421ed4b37932fab0bbdcfae45ede2e2"
server_key_version: "encryptionkey@0"
I20260926 16:37:11.878460 13600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:11.903728 13600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:11.907245 13600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:11.907411 13600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:11.916679 13600 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:40583
I20260926 16:37:11.916721 13677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:40583 every 8 connection(s)
I20260926 16:37:11.917804 13600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:11.918886 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 13600
I20260926 16:37:11.918977 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:11.919261 11898 external_mini_cluster.cc:1428] Setting key d5fc78e557b02a450181e846423a73d4
I20260926 16:37:11.921121 13678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:11.928030 13678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Bootstrap starting.
Sep 26 16:37:11 dist-test-slave-p39x krb5kdc[13587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:11.930377 13678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:11.931119 13678 log.cc:824] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:11.933211 13678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: No bootstrap required, opened a new log
I20260926 16:37:11.935323 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:11.920786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.11.158.190:40583)
0926 16:37:11.921223 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:11.921241 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:11.921286 (+ 45us) server_negotiation.cc:407] Connection header received
0926 16:37:11.921936 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:11.921953 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:11.922291 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:11.922579 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:11.923484 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:11.924771 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:11.925963 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:11.926434 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:11.926556 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:11.929582 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:11.929588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:11.929601 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:11.929635 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:11.932246 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:11.932786 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:11.932801 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:11.932806 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:11.932926 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:11.933288 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:11.933297 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:11.933319 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:11.933755 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:11.933979 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:11.934294 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 16:37:11.934604 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":149,"threads_started":1}
I20260926 16:37:11.936403 13678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0532e93e69f64d81a517db3adb4a14c8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40583 } }
I20260926 16:37:11.936661 13678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:11.936738 13678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0532e93e69f64d81a517db3adb4a14c8, State: Initialized, Role: FOLLOWER
I20260926 16:37:11.937203 13678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0532e93e69f64d81a517db3adb4a14c8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40583 } }
I20260926 16:37:11.937354 13678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:11.937440 13678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:11.937543 13678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:11.938452 13678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0532e93e69f64d81a517db3adb4a14c8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40583 } }
I20260926 16:37:11.938820 13678 leader_election.cc:302] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0532e93e69f64d81a517db3adb4a14c8; no voters:
I20260926 16:37:11.939146 13678 leader_election.cc:288] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:11.939285 13683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:11.939529 13683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [term 1 LEADER]: Becoming Leader. State: Replica: 0532e93e69f64d81a517db3adb4a14c8, State: Running, Role: LEADER
I20260926 16:37:11.939843 13683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0532e93e69f64d81a517db3adb4a14c8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40583 } }
I20260926 16:37:11.940522 13678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:11.941748 13684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0532e93e69f64d81a517db3adb4a14c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0532e93e69f64d81a517db3adb4a14c8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40583 } } }
I20260926 16:37:11.941875 13684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:11.942144 13685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0532e93e69f64d81a517db3adb4a14c8. Latest consensus state: current_term: 1 leader_uuid: "0532e93e69f64d81a517db3adb4a14c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0532e93e69f64d81a517db3adb4a14c8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40583 } } }
I20260926 16:37:11.942233 13685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:11.942348 13692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:11.945201 13692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:11.950755 13692 catalog_manager.cc:1347] Generated new cluster ID: 40f97d6b40a64d339fbc9d00c6b17e1f
I20260926 16:37:11.950829 13692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:11.958151 13692 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:11.959383 13692 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:11.965549 13692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Generated new TSK 0
I20260926 16:37:11.966354 13692 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:11.999394 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40583
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:12.122387 13704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:12.122654 13704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:12.122704 13704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:12.126732 13704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:12.126794 13704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:12.126834 13704 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 16:37:12.126849 13704 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:12.126883 13704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:12.132022 13704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:40583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:12.133113 13704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:12.134107 13704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:12.144251 13710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:12.144233 13709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:12.144227 13712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:12.144325 13704 server_base.cc:1034] running on GCE node
I20260926 16:37:12.145056 13704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:12.145744 13704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:12.146992 13704 hybrid_clock.cc:648] HybridClock initialized: now 1790440632146966 us; error 38 us; skew 500 ppm
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:12.150260 13704 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:12.151289 13704 webserver.cc:460] Webserver started at http://127.11.158.129:34693/ using document root <none> and password file <none>
I20260926 16:37:12.151885 13704 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:12.151953 13704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:12.152180 13704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:12.154371 13704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "756a8b88bf38475e913cdb55e301ce54"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "22047a1a35e5bc4476edc51d4c3cd659"
server_key_iv: "3aa85a5ab040ea23756e35b8036fc450"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.154944 13704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "756a8b88bf38475e913cdb55e301ce54"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "22047a1a35e5bc4476edc51d4c3cd659"
server_key_iv: "3aa85a5ab040ea23756e35b8036fc450"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.158790 13704 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:37:12.161438 13719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:12.162746 13704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:12.162894 13704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "756a8b88bf38475e913cdb55e301ce54"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "22047a1a35e5bc4476edc51d4c3cd659"
server_key_iv: "3aa85a5ab040ea23756e35b8036fc450"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.163015 13704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:12.182078 13704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:12.185658 13704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:12.185875 13704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:12.187469 13704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:12.187533 13704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:12.187583 13704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:12.187609 13704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:12.198647 13704 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:43807
I20260926 16:37:12.198776 13834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:43807 every 8 connection(s)
I20260926 16:37:12.199577 13704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:12.201994 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.188197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:33123 (local address 127.11.158.190:40583)
0926 16:37:12.188330 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.188357 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.189501 (+ 1144us) server_negotiation.cc:407] Connection header received
0926 16:37:12.190874 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.190877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.190959 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.191075 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:12.192855 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.193482 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.195031 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.195296 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.195411 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.198149 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:12.198154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:12.198161 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:12.198189 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:12.199926 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.200584 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.200588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.200589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.200686 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.200991 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.200993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.200994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.201174 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:12.201249 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.201750 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.201874 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:37:12.202755 13728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.188792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40583 (local address 127.11.158.129:33123)
0926 16:37:12.189308 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.189374 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.190579 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.191282 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.191295 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.191822 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:12.192676 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.192700 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.193686 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.193691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.194907 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.194917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.195452 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.195456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.195630 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.196378 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:12.196401 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:12.197974 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:12.200044 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.200050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.200065 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.200459 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.200773 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.200775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.200776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.200904 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.201266 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:12.201271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:12.201500 (+ 229us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.201671 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.201912 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":101,"threads_started":1}
I20260926 16:37:12.204007 13835 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40583
I20260926 16:37:12.204170 13835 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:12.204772 13835 heartbeater.cc:511] Master 127.11.158.190:40583 requested a full tablet report, sending...
I20260926 16:37:12.206442 13632 ts_manager.cc:194] Registered new tserver with Master: 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807)
I20260926 16:37:12.206660 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 13704
I20260926 16:37:12.206796 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:12.207054 11898 external_mini_cluster.cc:1428] Setting key 082e50301fcf966e5cc7ef376616fc73
I20260926 16:37:12.208010 13632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:33123
I20260926 16:37:12.214078 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.206212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:52759 (local address 127.11.158.190:40583)
0926 16:37:12.206334 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.206338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.206491 (+ 153us) server_negotiation.cc:407] Connection header received
0926 16:37:12.206633 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.206637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.206713 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.206765 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:12.207370 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.207991 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.209160 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.209575 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.209698 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.210859 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:12.210863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:12.210866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:12.210889 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:12.212486 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.212902 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.212907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.212909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.212994 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.213270 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.213275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.213277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.213449 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:12.213563 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.213802 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.213915 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:37:12.214114 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.206082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40583 (local address 127.11.158.129:52759)
0926 16:37:12.206398 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.206413 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.206514 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.206901 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.206904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.207132 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:12.207247 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.207255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.208239 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.208242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.209036 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.209046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.209704 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.209708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.209797 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.210386 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:12.210398 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:12.210716 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:12.212598 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.212603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.212605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.212805 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.213087 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.213092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.213094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.213181 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.213543 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:12.213546 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:12.213637 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.213753 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.213902 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:12.235610 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40583
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:12.350342 13844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:12.350694 13844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:12.350775 13844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:12.355019 13844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:12.355129 13844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:12.355206 13844 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 16:37:12.355252 13844 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:12.355319 13844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:12.360930 13844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:40583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:12.362106 13844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:12.363165 13844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:12.373107 13852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:12.373108 13849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:12.373172 13844 server_base.cc:1034] running on GCE node
W20260926 16:37:12.373191 13850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:12.373790 13844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:12.374413 13844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:12.375615 13844 hybrid_clock.cc:648] HybridClock initialized: now 1790440632375595 us; error 34 us; skew 500 ppm
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:12.378791 13844 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:12.379757 13844 webserver.cc:460] Webserver started at http://127.11.158.130:37679/ using document root <none> and password file <none>
I20260926 16:37:12.380440 13844 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:12.380549 13844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:12.380805 13844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:12.383322 13844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "2559e3f3d5e2471b8363521b1e707b4b"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "95bd59cbdb21a7dd645e65467ba2a106"
server_key_iv: "105b9b5b45d4d0df2136918933188ce5"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.383953 13844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "2559e3f3d5e2471b8363521b1e707b4b"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "95bd59cbdb21a7dd645e65467ba2a106"
server_key_iv: "105b9b5b45d4d0df2136918933188ce5"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.388130 13844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260926 16:37:12.391291 13859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:12.392771 13844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:12.392910 13844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "2559e3f3d5e2471b8363521b1e707b4b"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "95bd59cbdb21a7dd645e65467ba2a106"
server_key_iv: "105b9b5b45d4d0df2136918933188ce5"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.393034 13844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:12.414467 13844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:12.417923 13844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:12.418135 13844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:12.419627 13844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:12.419708 13844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:12.419780 13844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:12.419831 13844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:12.431058 13844 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:40333
I20260926 16:37:12.431198 13974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:40333 every 8 connection(s)
I20260926 16:37:12.432130 13844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:12.432540 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 13844
I20260926 16:37:12.432668 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:12.433002 11898 external_mini_cluster.cc:1428] Setting key bf9773e1f10b8df74e744f6c51888b2c
I20260926 16:37:12.434813 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.420682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:57267 (local address 127.11.158.190:40583)
0926 16:37:12.420820 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.420824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.421947 (+ 1123us) server_negotiation.cc:407] Connection header received
0926 16:37:12.423070 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.423073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.423134 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.423235 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:12.424914 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.425859 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.427305 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.427551 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.427669 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.430790 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:12.430794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:12.430797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:12.430827 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:12.432537 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.433140 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.433143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.433145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.433234 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.433602 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.433605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.433606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.433866 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:12.433972 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.434523 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.434655 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:37:12.435649 13868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.421299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40583 (local address 127.11.158.130:57267)
0926 16:37:12.421784 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.421827 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.422817 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.423442 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.423453 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.423954 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:12.424698 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.424719 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.426022 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.426027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.427166 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.427176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.427726 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.427730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.427912 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.428574 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:12.428592 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:12.430566 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:12.432690 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.432694 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.432708 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.433026 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.433337 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.433341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.433343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.433510 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.433985 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:12.433990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:12.434289 (+ 299us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.434463 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.434853 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260926 16:37:12.438964 13975 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40583
I20260926 16:37:12.439121 13975 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:12.439613 13975 heartbeater.cc:511] Master 127.11.158.190:40583 requested a full tablet report, sending...
I20260926 16:37:12.440703 13632 ts_manager.cc:194] Registered new tserver with Master: 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333)
I20260926 16:37:12.441437 13632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:57267
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:12.447646 13982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.439385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40583 (local address 127.11.158.130:56707)
0926 16:37:12.439688 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.439706 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.439822 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.440098 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.440103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.440387 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:12.440519 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.440527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.441642 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.441646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.442794 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.442808 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.443376 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.443379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.443448 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.444017 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:12.444029 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:12.444399 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:12.446097 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.446100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.446102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.446296 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.446593 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.446596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.446597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.446685 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.447197 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:12.447199 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:12.447279 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.447362 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.447469 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260926 16:37:12.447760 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.439178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:56707 (local address 127.11.158.190:40583)
0926 16:37:12.439292 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.439296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.439772 (+ 476us) server_negotiation.cc:407] Connection header received
0926 16:37:12.439928 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.439931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.439977 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.440064 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:12.440656 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.441452 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.442952 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.443265 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.443372 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.444531 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:12.444534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:12.444536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:12.444559 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:12.445983 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.446394 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.446400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.446404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.446499 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.446773 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.446779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.446783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.447096 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:12.447205 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.447454 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.447594 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:12.464638 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40583
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:12.590682 13984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:12.590941 13984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:12.590988 13984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:12.595016 13984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:12.595070 13984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:12.595111 13984 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 16:37:12.595125 13984 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:12.595158 13984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:12.600515 13984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:40583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:12.601490 13984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:12.602437 13984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:12.612048 13989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:12.612059 13992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:12.612064 13990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:12.612152 13984 server_base.cc:1034] running on GCE node
I20260926 16:37:12.612737 13984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:12.613338 13984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:12.614523 13984 hybrid_clock.cc:648] HybridClock initialized: now 1790440632614503 us; error 31 us; skew 500 ppm
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:12.617286 13984 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:12.618223 13984 webserver.cc:460] Webserver started at http://127.11.158.131:35973/ using document root <none> and password file <none>
I20260926 16:37:12.618820 13984 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:12.618914 13984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:12.619151 13984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:12.621337 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "5373b8f2f37fd60989344de83ebbc9d4"
server_key_iv: "ce33a27a23db5b4b629f003182fe33f8"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.621905 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "5373b8f2f37fd60989344de83ebbc9d4"
server_key_iv: "ce33a27a23db5b4b629f003182fe33f8"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.625699 13984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:37:12.628140 13999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:12.629558 13984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:37:12.629693 13984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a"
format_stamp: "Formatted at 2026-09-26 16:37:12 on dist-test-slave-p39x"
server_key: "5373b8f2f37fd60989344de83ebbc9d4"
server_key_iv: "ce33a27a23db5b4b629f003182fe33f8"
server_key_version: "encryptionkey@0"
I20260926 16:37:12.629827 13984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:12.650532 13984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:12.653971 13984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:12.654182 13984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:12.655660 13984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:12.655745 13984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:12.655814 13984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:12.655854 13984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:12.666164 13984 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:33333
I20260926 16:37:12.666275 14114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:33333 every 8 connection(s)
I20260926 16:37:12.667261 13984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:12.670362 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.656459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:44879 (local address 127.11.158.190:40583)
0926 16:37:12.656584 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.656588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.657633 (+ 1045us) server_negotiation.cc:407] Connection header received
0926 16:37:12.658778 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.658785 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.658856 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.658955 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:12.660514 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.661396 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.663225 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.663609 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.663732 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.666714 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:12.666718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:12.666721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:12.666749 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:12.668462 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.669011 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.669013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.669015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.669096 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.669404 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.669407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.669408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.669597 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:12.669691 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.670142 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.670246 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:37:12.671162 14008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.657040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40583 (local address 127.11.158.131:44879)
0926 16:37:12.657490 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.657530 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.658494 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.659177 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.659189 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.659616 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:12.660324 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.660378 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.661594 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.661597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.663109 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.663119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.663754 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.663757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.663937 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.664673 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:12.664701 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:12.666548 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:12.668593 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.668599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.668612 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.668902 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.669182 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.669184 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.669186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.669322 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.669700 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:12.669706 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:12.669931 (+ 225us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.670078 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.670337 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260926 16:37:12.671864 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 13984
I20260926 16:37:12.671988 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:12.672304 11898 external_mini_cluster.cc:1428] Setting key 795992d8d955fc23a31e67c21491e3fe
I20260926 16:37:12.672669 14115 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40583
I20260926 16:37:12.672822 14115 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:12.673432 14115 heartbeater.cc:511] Master 127.11.158.190:40583 requested a full tablet report, sending...
I20260926 16:37:12.674520 13631 ts_manager.cc:194] Registered new tserver with Master: b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333)
I20260926 16:37:12.675293 13631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:44879
I20260926 16:37:12.682029 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.674937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40583 (local address 127.11.158.131:57503)
0926 16:37:12.675677 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.675692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.675795 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.676039 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.676041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.676228 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:12.676321 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.676326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.677339 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.677342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.677938 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.677946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.678351 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.678354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.678426 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.678993 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:12.679005 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:12.679338 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:12.680789 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.680791 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.680793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.680981 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.681224 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:12.681226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:12.681228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:12.681298 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:12.681624 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:12.681626 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:12.681696 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.681775 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.681880 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":94,"threads_started":1}
I20260926 16:37:12.682029 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.675010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:57503 (local address 127.11.158.190:40583)
0926 16:37:12.675126 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.675130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.675771 (+ 641us) server_negotiation.cc:407] Connection header received
0926 16:37:12.675890 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.675893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.675937 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.676018 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:12.676482 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.677212 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.678040 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.678256 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.678334 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.679454 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:12.679457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:12.679459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:12.679477 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:12.680698 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.681061 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.681063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.681064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.681145 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.681380 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.681386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.681390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.681549 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:12.681624 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.681832 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.681921 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:37:12.686088 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:37:12 dist-test-slave-p39x krb5kdc[13587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:12.711470 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.703002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.11.158.190:40583)
0926 16:37:12.703148 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.703152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.703164 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:37:12.703217 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.703219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.703259 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.703355 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:12.704101 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.704700 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.705609 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.705831 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.705947 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.708436 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:12.708441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:12.708444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:12.708480 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:12.710016 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.710406 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.710408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.710410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.710481 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:12.710701 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:12.710703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:12.710704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:12.710869 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:12.710965 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.711161 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.711262 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:37:12.714331 13631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:12.717303 13631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:12.731901 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.725899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:33333 (local address 127.0.0.1:60424)
0926 16:37:12.726932 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.726989 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.727121 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.727610 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.727616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.727638 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:12.727788 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.727798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.729172 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.729175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.730288 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.730298 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.731338 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.731341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.731459 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.731497 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.731613 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.731735 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":87,"threads_started":1}
I20260926 16:37:12.731949 14134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.726055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60424 (local address 127.11.158.131:33333)
0926 16:37:12.727162 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.727175 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.727201 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:37:12.727285 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.727289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.727453 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.727611 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:12.727928 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.729006 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.730411 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.730985 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.731133 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.731274 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.731675 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.731800 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":113,"threads_started":1}
I20260926 16:37:12.732674 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.726506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:40333 (local address 127.0.0.1:49662)
0926 16:37:12.727878 (+ 1372us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.727893 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.728006 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.728476 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.728479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.728496 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:12.728583 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.728588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.729853 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.729856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.730969 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.730980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.732200 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.732204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.732360 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.732387 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.732459 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.732534 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1298,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260926 16:37:12.732857 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.726774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42076 (local address 127.11.158.129:43807)
0926 16:37:12.727576 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.727580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.727597 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:37:12.727658 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.727662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.727811 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.727971 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:12.728402 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.729529 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.731034 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.731805 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.731925 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.732058 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.732609 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.732726 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":76,"threads_started":1}
I20260926 16:37:12.733266 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.726686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.11.158.130:40333)
0926 16:37:12.727193 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:12.727201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:12.727991 (+ 790us) server_negotiation.cc:407] Connection header received
0926 16:37:12.728144 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:12.728148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:12.728293 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:12.728496 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:12.728695 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.729736 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.731094 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.731699 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.731851 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.732136 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:12.732862 (+ 726us) server_negotiation.cc:299] Negotiation successful
0926 16:37:12.733132 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":111,"threads_started":1}
I20260926 16:37:12.733502 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:12.726094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43807 (local address 127.0.0.1:42076)
0926 16:37:12.727308 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:12.727319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:12.727385 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:12.728109 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:12.728123 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:12.728140 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:12.728245 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.728252 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.729663 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.729666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.730888 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:12.730898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:12.732120 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:12.732126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:12.732252 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:12.732287 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 16:37:12.732433 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 16:37:12.732513 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":77,"threads_started":1}
I20260926 16:37:12.735425 14049 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4de8b7bda84decbbd58e52aac1a3e1 (DEFAULT_TABLE table=test-table [id=a20137dc0685449c835004f784ecd27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:12.736672 14049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4de8b7bda84decbbd58e52aac1a3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:12.736258 13769 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4de8b7bda84decbbd58e52aac1a3e1 (DEFAULT_TABLE table=test-table [id=a20137dc0685449c835004f784ecd27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:12.737344 13769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4de8b7bda84decbbd58e52aac1a3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:12.743085 13909 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4de8b7bda84decbbd58e52aac1a3e1 (DEFAULT_TABLE table=test-table [id=a20137dc0685449c835004f784ecd27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:12.743988 13909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4de8b7bda84decbbd58e52aac1a3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:12.744495 14138 tablet_bootstrap.cc:492] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Bootstrap starting.
I20260926 16:37:12.745883 14139 tablet_bootstrap.cc:492] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Bootstrap starting.
I20260926 16:37:12.746835 14138 tablet_bootstrap.cc:654] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:12.747756 14139 tablet_bootstrap.cc:654] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:12.747781 14138 log.cc:824] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:12.748600 14139 log.cc:824] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:12.749877 14138 tablet_bootstrap.cc:492] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: No bootstrap required, opened a new log
I20260926 16:37:12.750072 14138 ts_tablet_manager.cc:1397] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:12.750351 14139 tablet_bootstrap.cc:492] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: No bootstrap required, opened a new log
I20260926 16:37:12.750543 14139 ts_tablet_manager.cc:1397] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:37:12.751770 14142 tablet_bootstrap.cc:492] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Bootstrap starting.
I20260926 16:37:12.754094 14139 raft_consensus.cc:348] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:12.754109 14138 raft_consensus.cc:348] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:12.754365 14138 raft_consensus.cc:374] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:12.754365 14139 raft_consensus.cc:374] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:12.754419 14139 raft_consensus.cc:729] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Initialized, Role: FOLLOWER
I20260926 16:37:12.754427 14138 raft_consensus.cc:729] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756a8b88bf38475e913cdb55e301ce54, State: Initialized, Role: FOLLOWER
I20260926 16:37:12.754792 14142 tablet_bootstrap.cc:654] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:12.754885 14139 consensus_queue.cc:260] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:12.754938 14138 consensus_queue.cc:260] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:12.755529 14115 heartbeater.cc:503] Master 127.11.158.190:40583 was elected leader, sending a full tablet report...
I20260926 16:37:12.755609 14142 log.cc:824] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:12.755868 14138 ts_tablet_manager.cc:1428] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:37:12.755926 14139 ts_tablet_manager.cc:1428] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:37:12.756170 13835 heartbeater.cc:503] Master 127.11.158.190:40583 was elected leader, sending a full tablet report...
I20260926 16:37:12.757455 14142 tablet_bootstrap.cc:492] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: No bootstrap required, opened a new log
I20260926 16:37:12.757648 14142 ts_tablet_manager.cc:1397] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:12.760742 14142 raft_consensus.cc:348] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:12.760968 14142 raft_consensus.cc:374] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:12.761040 14142 raft_consensus.cc:729] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2559e3f3d5e2471b8363521b1e707b4b, State: Initialized, Role: FOLLOWER
I20260926 16:37:12.761467 14142 consensus_queue.cc:260] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:12.762105 13975 heartbeater.cc:503] Master 127.11.158.190:40583 was elected leader, sending a full tablet report...
I20260926 16:37:12.762396 14142 ts_tablet_manager.cc:1428] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260926 16:37:12.918325 14116 tablet.cc:2369] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:12.933740 13976 tablet.cc:2369] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:12.951176 13836 tablet.cc:2369] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:13.070884 14143 raft_consensus.cc:482] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:13.071137 14143 raft_consensus.cc:504] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:13.072369 14143 leader_election.cc:288] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333)
I20260926 16:37:13.076650 14008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.072656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43807 (local address 127.11.158.131:43197)
0926 16:37:13.072865 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.072881 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.072987 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.073292 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.073295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.073312 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:13.073458 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.073465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.074296 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.074298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.075427 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.075438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.076205 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.076208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.076292 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.076311 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.076427 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.076489 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":21}
I20260926 16:37:13.076697 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.072743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:43197 (local address 127.11.158.129:43807)
0926 16:37:13.072881 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.072886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.072947 (+ 61us) server_negotiation.cc:407] Connection header received
0926 16:37:13.073103 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.073106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.073170 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.073262 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:13.073613 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.074188 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.075576 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.076093 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.076209 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.076261 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.076462 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.076563 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:37:13.077093 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.072742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:52225 (local address 127.11.158.130:40333)
0926 16:37:13.072903 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.072908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.073183 (+ 275us) server_negotiation.cc:407] Connection header received
0926 16:37:13.073255 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.073258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.073301 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.073396 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:13.074072 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.074623 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.075867 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.076462 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.076575 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.076617 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.076857 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.076948 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:37:13.077351 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4de8b7bda84decbbd58e52aac1a3e1" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54" is_pre_election: true
I20260926 16:37:13.077414 14147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.072659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:40333 (local address 127.11.158.131:52225)
0926 16:37:13.073075 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.073091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.073184 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.073408 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.073411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.073428 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:13.073941 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.073950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.074738 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.074742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.075759 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.075766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.076598 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.076602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.076700 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.076725 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.077239 (+ 514us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.077309 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":60,"thread_start_us":145,"threads_started":1}
I20260926 16:37:13.077689 13789 raft_consensus.cc:2454] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 0.
I20260926 16:37:13.077996 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4de8b7bda84decbbd58e52aac1a3e1" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b" is_pre_election: true
I20260926 16:37:13.078204 14001 leader_election.cc:302] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756a8b88bf38475e913cdb55e301ce54, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:13.078315 13929 raft_consensus.cc:2454] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 0.
I20260926 16:37:13.078485 14143 raft_consensus.cc:2781] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:13.078589 14143 raft_consensus.cc:482] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:13.078649 14143 raft_consensus.cc:3037] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.079676 14143 raft_consensus.cc:504] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:13.080026 14143 leader_election.cc:288] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 election: Requested vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333)
I20260926 16:37:13.080441 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4de8b7bda84decbbd58e52aac1a3e1" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b"
I20260926 16:37:13.080446 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4de8b7bda84decbbd58e52aac1a3e1" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54"
I20260926 16:37:13.080576 13929 raft_consensus.cc:3037] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.080576 13789 raft_consensus.cc:3037] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.081833 13789 raft_consensus.cc:2454] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 1.
I20260926 16:37:13.081832 13929 raft_consensus.cc:2454] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 1.
I20260926 16:37:13.082144 14000 leader_election.cc:302] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:13.082352 14143 raft_consensus.cc:2781] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:13.082643 14143 raft_consensus.cc:686] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 LEADER]: Becoming Leader. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Running, Role: LEADER
I20260926 16:37:13.082973 14143 consensus_queue.cc:237] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } }
I20260926 16:37:13.086334 13631 catalog_manager.cc:5495] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a reported cstate change: term changed from 0 to 1, leader changed from <none> to b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131). New cstate: current_term: 1 leader_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } health_report { overall_health: UNKNOWN } } }
Sep 26 16:37:13 dist-test-slave-p39x krb5kdc[13587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440631, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:13.202527 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.194046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40583 (local address 127.0.0.1:49304)
0926 16:37:13.194374 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.194386 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.194521 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.194784 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.194787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.195200 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:13.195286 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.195292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.196035 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.196041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.196687 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.196696 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.197136 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.197139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.197257 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.197784 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:13.197796 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:13.199198 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:13.200979 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:13.200986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:13.200990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:13.201207 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:13.201475 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:13.201477 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:13.201478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:13.201571 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:13.202032 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:13.202038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:13.202176 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.202286 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.202397 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":17,"thread_start_us":116,"threads_started":1}
I20260926 16:37:13.202651 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.194084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.11.158.190:40583)
0926 16:37:13.194230 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.194238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.194462 (+ 224us) server_negotiation.cc:407] Connection header received
0926 16:37:13.194636 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.194639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.194680 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.194765 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:13.195428 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.195933 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.196801 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.197010 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.197127 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.199338 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:13.199342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:13.199345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:13.199373 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:13.200882 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:13.201301 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:13.201304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:13.201306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:13.201389 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:13.201659 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:13.201662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:13.201664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:13.201902 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:13.202003 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.202363 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.202475 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":45}
I20260926 16:37:13.206586 13631 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49304:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 16:37:13.207294 13631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:13.211979 13769 tablet_service.cc:1467] Processing CreateTablet for tablet c6662d5a90f54f188e60a944e8304b43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=066fc44ff0e14adba937fab0ead789f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:13.212069 13909 tablet_service.cc:1467] Processing CreateTablet for tablet c6662d5a90f54f188e60a944e8304b43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=066fc44ff0e14adba937fab0ead789f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:13.212242 14049 tablet_service.cc:1467] Processing CreateTablet for tablet c6662d5a90f54f188e60a944e8304b43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=066fc44ff0e14adba937fab0ead789f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:13.212708 13769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6662d5a90f54f188e60a944e8304b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:13.212775 14049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6662d5a90f54f188e60a944e8304b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:13.212775 13909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6662d5a90f54f188e60a944e8304b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:13.214956 14138 tablet_bootstrap.cc:492] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: Bootstrap starting.
I20260926 16:37:13.215514 14142 tablet_bootstrap.cc:492] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: Bootstrap starting.
I20260926 16:37:13.215772 14138 tablet_bootstrap.cc:654] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:13.216504 14142 tablet_bootstrap.cc:654] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:13.217101 14138 tablet_bootstrap.cc:492] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: No bootstrap required, opened a new log
I20260926 16:37:13.217128 14139 tablet_bootstrap.cc:492] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Bootstrap starting.
I20260926 16:37:13.217190 14138 ts_tablet_manager.cc:1397] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:13.217643 14138 raft_consensus.cc:348] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.217763 14138 raft_consensus.cc:374] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:13.217746 14142 tablet_bootstrap.cc:492] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: No bootstrap required, opened a new log
I20260926 16:37:13.217797 14138 raft_consensus.cc:729] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756a8b88bf38475e913cdb55e301ce54, State: Initialized, Role: FOLLOWER
I20260926 16:37:13.217849 14142 ts_tablet_manager.cc:1397] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 16:37:13.217924 14138 consensus_queue.cc:260] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.218011 14139 tablet_bootstrap.cc:654] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:13.218231 14138 ts_tablet_manager.cc:1428] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:13.218263 14142 raft_consensus.cc:348] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.218400 14142 raft_consensus.cc:374] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:13.218442 14142 raft_consensus.cc:729] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2559e3f3d5e2471b8363521b1e707b4b, State: Initialized, Role: FOLLOWER
I20260926 16:37:13.218586 14142 consensus_queue.cc:260] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.218873 14142 ts_tablet_manager.cc:1428] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:37:13.218847 14146 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.218971 14146 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:13.219105 14144 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.219234 14144 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:13.220075 14139 tablet_bootstrap.cc:492] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: No bootstrap required, opened a new log
I20260926 16:37:13.220160 14139 ts_tablet_manager.cc:1397] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 16:37:13.220650 14139 raft_consensus.cc:348] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.220760 14139 raft_consensus.cc:374] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:13.220798 14139 raft_consensus.cc:729] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Initialized, Role: FOLLOWER
I20260926 16:37:13.220927 14139 consensus_queue.cc:260] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.221228 14139 ts_tablet_manager.cc:1428] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:13.221522 14143 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.221624 14143 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:13.250489 14146 raft_consensus.cc:482] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:13.250664 14146 raft_consensus.cc:504] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.251799 14146 leader_election.cc:288] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333)
I20260926 16:37:13.256132 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.252191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:52107 (local address 127.11.158.131:33333)
0926 16:37:13.252486 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.252502 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.252594 (+ 92us) server_negotiation.cc:407] Connection header received
0926 16:37:13.252766 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.252769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.252808 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.252901 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:13.253224 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.253757 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.255116 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.255527 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.255667 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.255725 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.255906 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.255988 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":88,"threads_started":1}
I20260926 16:37:13.256151 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.252110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:33333 (local address 127.11.158.130:52107)
0926 16:37:13.252485 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.252502 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.252645 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.252911 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.252914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.252929 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:13.253108 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.253114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.253867 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.253873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.254982 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.254991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.255666 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.255670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.255770 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.255792 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.255889 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.256007 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":69,"thread_start_us":99,"threads_started":1}
I20260926 16:37:13.256310 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.252192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:59701 (local address 127.11.158.129:43807)
0926 16:37:13.252364 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.252371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.252664 (+ 293us) server_negotiation.cc:407] Connection header received
0926 16:37:13.252817 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.252820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.252871 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.252948 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:13.253582 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.254228 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.255307 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.255726 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.255857 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.255907 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.256093 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.256165 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:37:13.256740 14160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.252110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43807 (local address 127.11.158.130:59701)
0926 16:37:13.252558 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.252570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.252711 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.252969 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.252972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.252987 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:13.253465 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.253471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.254322 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.254329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.255184 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.255193 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.255843 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.255847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.255965 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.255981 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.256066 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.256132 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":67,"thread_start_us":66,"threads_started":1}
I20260926 16:37:13.256843 14069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6662d5a90f54f188e60a944e8304b43" candidate_uuid: "2559e3f3d5e2471b8363521b1e707b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" is_pre_election: true
I20260926 16:37:13.257090 14069 raft_consensus.cc:2454] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2559e3f3d5e2471b8363521b1e707b4b in term 0.
I20260926 16:37:13.257212 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6662d5a90f54f188e60a944e8304b43" candidate_uuid: "2559e3f3d5e2471b8363521b1e707b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54" is_pre_election: true
I20260926 16:37:13.257371 13789 raft_consensus.cc:2454] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2559e3f3d5e2471b8363521b1e707b4b in term 0.
I20260926 16:37:13.257550 13863 leader_election.cc:302] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:13.257843 14146 raft_consensus.cc:2781] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:13.257905 14146 raft_consensus.cc:482] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:13.257941 14146 raft_consensus.cc:3037] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.258898 14146 raft_consensus.cc:504] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.259220 14146 leader_election.cc:288] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [CANDIDATE]: Term 1 election: Requested vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333)
I20260926 16:37:13.259542 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6662d5a90f54f188e60a944e8304b43" candidate_uuid: "2559e3f3d5e2471b8363521b1e707b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54"
I20260926 16:37:13.259542 14069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6662d5a90f54f188e60a944e8304b43" candidate_uuid: "2559e3f3d5e2471b8363521b1e707b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a"
I20260926 16:37:13.259663 13789 raft_consensus.cc:3037] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.259654 14069 raft_consensus.cc:3037] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.260653 14069 raft_consensus.cc:2454] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2559e3f3d5e2471b8363521b1e707b4b in term 1.
I20260926 16:37:13.260654 13789 raft_consensus.cc:2454] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2559e3f3d5e2471b8363521b1e707b4b in term 1.
I20260926 16:37:13.260905 13863 leader_election.cc:302] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:13.261080 14146 raft_consensus.cc:2781] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:13.261270 14146 raft_consensus.cc:686] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [term 1 LEADER]: Becoming Leader. State: Replica: 2559e3f3d5e2471b8363521b1e707b4b, State: Running, Role: LEADER
I20260926 16:37:13.261668 14146 consensus_queue.cc:237] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.263475 14162 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: TxnStatusTablet state changed. Reason: New leader 2559e3f3d5e2471b8363521b1e707b4b. Latest consensus state: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.263582 14162 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:13.263844 14164 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 16:37:13.264041 14164 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 16:37:13.264787 13632 catalog_manager.cc:5495] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130). New cstate: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:13.288389 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.283870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.11.158.130:40333)
0926 16:37:13.284007 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.284013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.284075 (+ 62us) server_negotiation.cc:407] Connection header received
0926 16:37:13.284662 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.284666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.284734 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.284839 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:13.285092 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.285875 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.286741 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.287079 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.287181 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.287429 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.287982 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.288096 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.288164 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.288218 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:37:13.288403 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.283730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:40333 (local address 127.0.0.1:49664)
0926 16:37:13.284007 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.284020 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.284330 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.284874 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.284876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.284883 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:13.284981 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.284986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.286000 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.286003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.286629 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.286638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.287185 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.287187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.287256 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.287299 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.288083 (+ 784us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.288096 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.288152 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.288225 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260926 16:37:13.293395 14069 raft_consensus.cc:1261] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Refusing update from remote peer 2559e3f3d5e2471b8363521b1e707b4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:13.293603 13789 raft_consensus.cc:1261] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Refusing update from remote peer 2559e3f3d5e2471b8363521b1e707b4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:13.294090 14162 consensus_queue.cc:1035] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:13.294332 14146 consensus_queue.cc:1035] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:13.297562 14143 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: TxnStatusTablet state changed. Reason: New leader 2559e3f3d5e2471b8363521b1e707b4b. Latest consensus state: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.297698 14143 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:13.297834 14144 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: TxnStatusTablet state changed. Reason: New leader 2559e3f3d5e2471b8363521b1e707b4b. Latest consensus state: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.297963 14144 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:13.302060 14167 mvcc.cc:205] Tried to move back new op lower bound from 7333644833963094016 to 7333644833842368512. Current Snapshot: MvccSnapshot[applied={T|T < 7333644833963094016}]
I20260926 16:37:13.302491 14146 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.302618 14146 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:13.302675 14144 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.302769 14144 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P 756a8b88bf38475e913cdb55e301ce54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:13.303229 14169 mvcc.cc:205] Tried to move back new op lower bound from 7333644833963094016 to 7333644833842368512. Current Snapshot: MvccSnapshot[applied={T|T < 7333644833963094016}]
I20260926 16:37:13.303995 14162 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.304095 14162 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P 2559e3f3d5e2471b8363521b1e707b4b: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:13.304981 14143 tablet_replica.cc:439] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2559e3f3d5e2471b8363521b1e707b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } } }
I20260926 16:37:13.305091 14143 tablet_replica.cc:442] T c6662d5a90f54f188e60a944e8304b43 P b9a74d8eb1ae41b1accb18e2c9bfb41a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:13.314047 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.310344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.11.158.131:33333)
0926 16:37:13.310469 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.310474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.310680 (+ 206us) server_negotiation.cc:407] Connection header received
0926 16:37:13.310862 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.310864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.310914 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.311018 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:13.311268 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.311773 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.312689 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.312910 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.313021 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.313283 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.313667 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.313798 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.313858 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.313912 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:13.324812 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.321310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:40333 (local address 127.11.158.131:42875)
0926 16:37:13.321592 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.321616 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.321738 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.321990 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.321993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.322000 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:13.322134 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.322141 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.322875 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.322880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.323525 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.323533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.323990 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.323993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.324096 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.324139 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.324515 (+ 376us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.324524 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.324589 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.324666 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
I20260926 16:37:13.324796 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.321415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:42875 (local address 127.11.158.130:40333)
0926 16:37:13.321576 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.321580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.321683 (+ 103us) server_negotiation.cc:407] Connection header received
0926 16:37:13.321840 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.321843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.321899 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.321980 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:13.322242 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.322765 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.323646 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.323881 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.323989 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.324302 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.324425 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.324511 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.324598 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.324647 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 16:37:13.332456 13789 raft_consensus.cc:1261] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Refusing update from remote peer b9a74d8eb1ae41b1accb18e2c9bfb41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:13.332633 13929 raft_consensus.cc:1261] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Refusing update from remote peer b9a74d8eb1ae41b1accb18e2c9bfb41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:13.333010 14148 consensus_queue.cc:1035] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:13.333189 14143 consensus_queue.cc:1035] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:13.336849 14168 mvcc.cc:205] Tried to move back new op lower bound from 7333644834125107200 to 7333644833110355968. Current Snapshot: MvccSnapshot[applied={T|T < 7333644834125107200}]
I20260926 16:37:13.359485 14186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.355664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:33333 (local address 127.11.158.130:44517)
0926 16:37:13.356050 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.356067 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.356172 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.356452 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.356455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.356463 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:13.356598 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.356603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.357324 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.357330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.358023 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.358035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.358600 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.358605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.358701 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.358769 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.359201 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.359210 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.359281 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.359360 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260926 16:37:13.359483 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.355746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:44517 (local address 127.11.158.131:33333)
0926 16:37:13.355855 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.355859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.356145 (+ 286us) server_negotiation.cc:407] Connection header received
0926 16:37:13.356270 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.356273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.356326 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.356429 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:13.356710 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.357220 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.358159 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.358462 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.358576 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.358910 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.358999 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.359076 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.359294 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.359343 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:37:13.384292 13632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:13.384593 13632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:13.387485 13632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Sending DeleteTablet for 3 replicas of tablet 8e4de8b7bda84decbbd58e52aac1a3e1
I20260926 16:37:13.388478 13769 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4de8b7bda84decbbd58e52aac1a3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:13 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:42076
I20260926 16:37:13.389067 14049 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4de8b7bda84decbbd58e52aac1a3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:13 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60424
I20260926 16:37:13.389175 13909 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4de8b7bda84decbbd58e52aac1a3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:13 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49662
I20260926 16:37:13.389545 14191 tablet_replica.cc:330] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: stopping tablet replica
I20260926 16:37:13.389889 14191 raft_consensus.cc:2229] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:13.390285 14191 raft_consensus.cc:2258] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:13.390620 13632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:13.391074 13632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:13.391744 14191 ts_tablet_manager.cc:1905] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:13.393257 14190 tablet_replica.cc:330] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: stopping tablet replica
I20260926 16:37:13.393709 14190 raft_consensus.cc:2229] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:13.394043 14190 raft_consensus.cc:2258] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:13.395474 14190 ts_tablet_manager.cc:1905] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:13.396560 14192 tablet_replica.cc:330] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: stopping tablet replica
I20260926 16:37:13.396916 14192 raft_consensus.cc:2229] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:13.397089 14192 raft_consensus.cc:2258] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:13.397686 13769 tablet_service.cc:1467] Processing CreateTablet for tablet 0c24ff219aec4caa9ac3288785abff3b (DEFAULT_TABLE table=test-table [id=64f805d371c643b2a961637578847257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:13.398300 13909 tablet_service.cc:1467] Processing CreateTablet for tablet 0c24ff219aec4caa9ac3288785abff3b (DEFAULT_TABLE table=test-table [id=64f805d371c643b2a961637578847257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:13.398522 14192 ts_tablet_manager.cc:1905] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:13.398571 13909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c24ff219aec4caa9ac3288785abff3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:13.399554 14191 ts_tablet_manager.cc:1918] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:13.399646 14191 log.cc:1196] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/8e4de8b7bda84decbbd58e52aac1a3e1
I20260926 16:37:13.399971 14191 ts_tablet_manager.cc:1939] T 8e4de8b7bda84decbbd58e52aac1a3e1 P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting consensus metadata
I20260926 16:37:13.401284 13619 catalog_manager.cc:4841] TS b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333): tablet 8e4de8b7bda84decbbd58e52aac1a3e1 (table test-table [id=a20137dc0685449c835004f784ecd27c]) successfully deleted
I20260926 16:37:13.401453 14049 tablet_service.cc:1467] Processing CreateTablet for tablet 0c24ff219aec4caa9ac3288785abff3b (DEFAULT_TABLE table=test-table [id=64f805d371c643b2a961637578847257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:13.401710 14049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c24ff219aec4caa9ac3288785abff3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:13.404119 14139 tablet_bootstrap.cc:492] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: Bootstrap starting.
I20260926 16:37:13.405167 14139 tablet_bootstrap.cc:654] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:13.406600 14139 tablet_bootstrap.cc:492] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: No bootstrap required, opened a new log
I20260926 16:37:13.406692 14139 ts_tablet_manager.cc:1397] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:13.406765 14192 ts_tablet_manager.cc:1918] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:13.406863 14192 log.cc:1196] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/8e4de8b7bda84decbbd58e52aac1a3e1
I20260926 16:37:13.407128 14139 raft_consensus.cc:348] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.407228 14192 ts_tablet_manager.cc:1939] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 2559e3f3d5e2471b8363521b1e707b4b: Deleting consensus metadata
I20260926 16:37:13.407258 14139 raft_consensus.cc:374] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:13.407302 14139 raft_consensus.cc:729] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Initialized, Role: FOLLOWER
I20260926 16:37:13.407430 14139 consensus_queue.cc:260] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.408442 13616 catalog_manager.cc:4841] TS 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333): tablet 8e4de8b7bda84decbbd58e52aac1a3e1 (table test-table [id=a20137dc0685449c835004f784ecd27c]) successfully deleted
I20260926 16:37:13.408932 14142 tablet_bootstrap.cc:492] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: Bootstrap starting.
I20260926 16:37:13.410398 14142 tablet_bootstrap.cc:654] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:13.412011 14142 tablet_bootstrap.cc:492] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: No bootstrap required, opened a new log
I20260926 16:37:13.412101 14142 ts_tablet_manager.cc:1397] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260926 16:37:13.412695 14142 raft_consensus.cc:348] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.412827 14142 raft_consensus.cc:374] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:13.412863 14142 raft_consensus.cc:729] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2559e3f3d5e2471b8363521b1e707b4b, State: Initialized, Role: FOLLOWER
I20260926 16:37:13.415186 14139 ts_tablet_manager.cc:1428] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260926 16:37:13.416919 14142 consensus_queue.cc:260] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.417398 14142 ts_tablet_manager.cc:1428] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260926 16:37:13.417939 13769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c24ff219aec4caa9ac3288785abff3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:13.420805 14190 ts_tablet_manager.cc:1918] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:13.420979 14190 log.cc:1196] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/8e4de8b7bda84decbbd58e52aac1a3e1
I20260926 16:37:13.421392 14190 ts_tablet_manager.cc:1939] T 8e4de8b7bda84decbbd58e52aac1a3e1 P 756a8b88bf38475e913cdb55e301ce54: Deleting consensus metadata
I20260926 16:37:13.422744 13617 catalog_manager.cc:4841] TS 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807): tablet 8e4de8b7bda84decbbd58e52aac1a3e1 (table test-table [id=a20137dc0685449c835004f784ecd27c]) successfully deleted
I20260926 16:37:13.423278 14138 tablet_bootstrap.cc:492] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: Bootstrap starting.
I20260926 16:37:13.424223 14138 tablet_bootstrap.cc:654] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:13.425483 14138 tablet_bootstrap.cc:492] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: No bootstrap required, opened a new log
I20260926 16:37:13.425565 14138 ts_tablet_manager.cc:1397] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:13.425987 14138 raft_consensus.cc:348] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.426107 14138 raft_consensus.cc:374] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:13.426138 14138 raft_consensus.cc:729] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756a8b88bf38475e913cdb55e301ce54, State: Initialized, Role: FOLLOWER
I20260926 16:37:13.426256 14138 consensus_queue.cc:260] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.426499 14138 ts_tablet_manager.cc:1428] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:13.431033 14144 raft_consensus.cc:482] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:13.431136 14144 raft_consensus.cc:504] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.432265 14144 leader_election.cc:288] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333), b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333)
I20260926 16:37:13.438791 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.433852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:55139 (local address 127.11.158.130:40333)
0926 16:37:13.433958 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.433962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.434202 (+ 240us) server_negotiation.cc:407] Connection header received
0926 16:37:13.434425 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.434428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.434489 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.434535 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:13.434966 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.435764 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.437422 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.438059 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.438171 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.438232 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.438588 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.438670 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:37:13.439385 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.432695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:40333 (local address 127.11.158.129:55139)
0926 16:37:13.434119 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.434135 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.434271 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.434719 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.434721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.434738 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:13.434846 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.434851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.435918 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.435923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.437248 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.437259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.438342 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.438345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.438437 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.438459 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.439158 (+ 699us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.439241 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"thread_start_us":147,"threads_started":1}
I20260926 16:37:13.439919 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c24ff219aec4caa9ac3288785abff3b" candidate_uuid: "756a8b88bf38475e913cdb55e301ce54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b" is_pre_election: true
I20260926 16:37:13.440060 13929 raft_consensus.cc:2454] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 756a8b88bf38475e913cdb55e301ce54 in term 0.
I20260926 16:37:13.440526 13720 leader_election.cc:302] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, 756a8b88bf38475e913cdb55e301ce54; no voters:
I20260926 16:37:13.440811 14144 raft_consensus.cc:2781] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:13.440856 14144 raft_consensus.cc:482] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:13.440891 14144 raft_consensus.cc:3037] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.442240 14144 raft_consensus.cc:504] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.442677 14144 leader_election.cc:288] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [CANDIDATE]: Term 1 election: Requested vote from peers 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333), b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333)
I20260926 16:37:13.443435 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c24ff219aec4caa9ac3288785abff3b" candidate_uuid: "756a8b88bf38475e913cdb55e301ce54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b"
I20260926 16:37:13.443554 13929 raft_consensus.cc:3037] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.444856 13929 raft_consensus.cc:2454] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 756a8b88bf38475e913cdb55e301ce54 in term 1.
I20260926 16:37:13.445250 13720 leader_election.cc:302] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, 756a8b88bf38475e913cdb55e301ce54; no voters:
I20260926 16:37:13.445457 14144 raft_consensus.cc:2781] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:13.445653 14144 raft_consensus.cc:686] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 1 LEADER]: Becoming Leader. State: Replica: 756a8b88bf38475e913cdb55e301ce54, State: Running, Role: LEADER
I20260926 16:37:13.445888 14144 consensus_queue.cc:237] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:13.448357 13631 catalog_manager.cc:5495] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129). New cstate: current_term: 1 leader_uuid: "756a8b88bf38475e913cdb55e301ce54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:13.457695 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.433537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:42513 (local address 127.11.158.131:33333)
0926 16:37:13.433651 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.433655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.446153 (+ 12498us) server_negotiation.cc:407] Connection header received
0926 16:37:13.446372 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.446375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.446427 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.446475 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:13.446885 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.447644 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.456267 (+ 8623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.456932 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.457031 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.457084 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.457465 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.457551 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:37:13.458148 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.433092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:33333 (local address 127.11.158.129:42513)
0926 16:37:13.446057 (+ 12965us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.446074 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.446232 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.446607 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.446610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.446627 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:13.446762 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.446768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.447798 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.447802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.449154 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.449167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.457196 (+ 8029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.457201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.457308 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.457338 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.457903 (+ 565us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.457995 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12886,"thread_start_us":94,"threads_started":1}
I20260926 16:37:13.458626 14069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c24ff219aec4caa9ac3288785abff3b" candidate_uuid: "756a8b88bf38475e913cdb55e301ce54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" is_pre_election: true
I20260926 16:37:13.458789 14069 raft_consensus.cc:2454] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 756a8b88bf38475e913cdb55e301ce54 in term 0.
I20260926 16:37:13.459015 14068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c24ff219aec4caa9ac3288785abff3b" candidate_uuid: "756a8b88bf38475e913cdb55e301ce54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a"
I20260926 16:37:13.459122 14068 raft_consensus.cc:3037] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:13.460304 14068 raft_consensus.cc:2454] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 756a8b88bf38475e913cdb55e301ce54 in term 1.
I20260926 16:37:13.470086 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.465785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.11.158.129:43807)
0926 16:37:13.465902 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.465906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.465921 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:37:13.466056 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.466059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.466132 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.466246 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:13.466457 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.467283 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.468402 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.468759 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.468874 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.469103 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.469715 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.469843 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.469876 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.469941 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:13.482151 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.477950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:36847 (local address 127.11.158.130:40333)
0926 16:37:13.478088 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.478093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.478322 (+ 229us) server_negotiation.cc:407] Connection header received
0926 16:37:13.478420 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.478423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.478500 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.478583 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:13.479137 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.479926 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.480895 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.481247 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.481356 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.481640 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.481728 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.481800 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.481969 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.482020 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:37:13.482209 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.477805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:40333 (local address 127.11.158.129:36847)
0926 16:37:13.478240 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.478253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.478344 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.478592 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.478595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.478601 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:13.479027 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.479032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.480069 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.480072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.480780 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.480790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.481381 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.481384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.481457 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.481508 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.481815 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.481824 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.481932 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.482009 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":96,"threads_started":1}
I20260926 16:37:13.489061 13929 raft_consensus.cc:1261] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Refusing update from remote peer 756a8b88bf38475e913cdb55e301ce54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:13.489074 14068 raft_consensus.cc:1261] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Refusing update from remote peer 756a8b88bf38475e913cdb55e301ce54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:13.489558 14144 consensus_queue.cc:1035] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:13.490000 14144 consensus_queue.cc:1035] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:13.509348 14186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.505437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43807 (local address 127.11.158.130:54667)
0926 16:37:13.505598 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:13.505614 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:13.505715 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:13.506054 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:13.506057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:13.506065 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:13.506163 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.506169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.507208 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.507212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.507957 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:13.507966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.508556 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.508560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:13.508651 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.508687 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.509001 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.509008 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:37:13.509080 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 16:37:13.509174 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 16:37:13.509773 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:13.505468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:54667 (local address 127.11.158.129:43807)
0926 16:37:13.505587 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:13.505591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:13.505701 (+ 110us) server_negotiation.cc:407] Connection header received
0926 16:37:13.505808 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:13.505811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:13.505862 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:13.505939 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:13.506294 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.507070 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.508083 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:13.508432 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:13.508550 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:13.508826 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:13.508909 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:13.508986 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:13.509259 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 16:37:13.509322 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260926 16:37:13.510485 14169 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 16:37:15.071686 13631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:15.071843 13631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:15.074431 13631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Sending DeleteTablet for 3 replicas of tablet 0c24ff219aec4caa9ac3288785abff3b
I20260926 16:37:15.075013 13769 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c24ff219aec4caa9ac3288785abff3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:15 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:42076
I20260926 16:37:15.075114 13909 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c24ff219aec4caa9ac3288785abff3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:15 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49662
I20260926 16:37:15.075384 14228 tablet_replica.cc:330] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: stopping tablet replica
I20260926 16:37:15.075510 14228 raft_consensus.cc:2229] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:15.075634 14229 tablet_replica.cc:330] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: stopping tablet replica
I20260926 16:37:15.075824 14229 raft_consensus.cc:2229] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:15.075876 14228 raft_consensus.cc:2258] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:15.075958 14229 raft_consensus.cc:2258] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:15.076465 14049 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c24ff219aec4caa9ac3288785abff3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:15 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60424
I20260926 16:37:15.076746 14229 ts_tablet_manager.cc:1905] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:15.078433 14228 ts_tablet_manager.cc:1905] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:15.078799 14230 tablet_replica.cc:330] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: stopping tablet replica
I20260926 16:37:15.078931 14230 raft_consensus.cc:2229] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:15.079072 14230 raft_consensus.cc:2258] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:15.079622 14229 ts_tablet_manager.cc:1918] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:37:15.079685 14230 ts_tablet_manager.cc:1905] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:15.079728 14229 log.cc:1196] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/0c24ff219aec4caa9ac3288785abff3b
I20260926 16:37:15.079974 14229 ts_tablet_manager.cc:1939] T 0c24ff219aec4caa9ac3288785abff3b P 2559e3f3d5e2471b8363521b1e707b4b: Deleting consensus metadata
I20260926 16:37:15.080924 13616 catalog_manager.cc:4841] TS 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333): tablet 0c24ff219aec4caa9ac3288785abff3b (table test-table [id=64f805d371c643b2a961637578847257]) successfully deleted
I20260926 16:37:15.081154 14228 ts_tablet_manager.cc:1918] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:37:15.081235 14228 log.cc:1196] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/0c24ff219aec4caa9ac3288785abff3b
I20260926 16:37:15.081528 14228 ts_tablet_manager.cc:1939] T 0c24ff219aec4caa9ac3288785abff3b P 756a8b88bf38475e913cdb55e301ce54: Deleting consensus metadata
I20260926 16:37:15.082324 13617 catalog_manager.cc:4841] TS 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807): tablet 0c24ff219aec4caa9ac3288785abff3b (table test-table [id=64f805d371c643b2a961637578847257]) successfully deleted
I20260926 16:37:15.082794 14230 ts_tablet_manager.cc:1918] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:37:15.082890 14230 log.cc:1196] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/0c24ff219aec4caa9ac3288785abff3b
I20260926 16:37:15.083158 14230 ts_tablet_manager.cc:1939] T 0c24ff219aec4caa9ac3288785abff3b P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting consensus metadata
I20260926 16:37:15.083976 13619 catalog_manager.cc:4841] TS b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333): tablet 0c24ff219aec4caa9ac3288785abff3b (table test-table [id=64f805d371c643b2a961637578847257]) successfully deleted
Sep 26 16:37:15 dist-test-slave-p39x krb5kdc[13587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440635, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:37:15 dist-test-slave-p39x krb5kdc[13587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440635, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:15.106882 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:15.097488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.11.158.190:40583)
0926 16:37:15.097765 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:15.097775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:15.097826 (+ 51us) server_negotiation.cc:407] Connection header received
0926 16:37:15.098012 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:15.098016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:15.098070 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:15.098170 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:15.098889 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:15.099449 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:15.100312 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:15.100622 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:15.100744 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:15.103393 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:15.103396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:15.103399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:15.103436 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:15.105189 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:15.105692 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:15.105695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:15.105696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:15.105776 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:15.106030 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:15.106032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:15.106033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:15.106212 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:15.106312 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:15.106533 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 16:37:15.106655 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
I20260926 16:37:15.109719 13631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:15.110167 13631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:15.115106 13769 tablet_service.cc:1467] Processing CreateTablet for tablet 2a949613dc904b6c9db218b4b50d843d (DEFAULT_TABLE table=test-table [id=62b375a3f03d47fe9a60743bb78cc2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:15.115149 13909 tablet_service.cc:1467] Processing CreateTablet for tablet 2a949613dc904b6c9db218b4b50d843d (DEFAULT_TABLE table=test-table [id=62b375a3f03d47fe9a60743bb78cc2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:15.115176 14049 tablet_service.cc:1467] Processing CreateTablet for tablet 2a949613dc904b6c9db218b4b50d843d (DEFAULT_TABLE table=test-table [id=62b375a3f03d47fe9a60743bb78cc2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:15.115455 13909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a949613dc904b6c9db218b4b50d843d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:15.115460 13769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a949613dc904b6c9db218b4b50d843d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:15.115454 14049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a949613dc904b6c9db218b4b50d843d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:15.117754 14239 tablet_bootstrap.cc:492] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: Bootstrap starting.
I20260926 16:37:15.118342 14240 tablet_bootstrap.cc:492] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: Bootstrap starting.
I20260926 16:37:15.118786 14239 tablet_bootstrap.cc:654] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:15.119388 14240 tablet_bootstrap.cc:654] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:15.119518 14241 tablet_bootstrap.cc:492] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Bootstrap starting.
I20260926 16:37:15.120158 14239 tablet_bootstrap.cc:492] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: No bootstrap required, opened a new log
I20260926 16:37:15.120289 14239 ts_tablet_manager.cc:1397] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:15.120428 14241 tablet_bootstrap.cc:654] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:15.120653 14240 tablet_bootstrap.cc:492] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: No bootstrap required, opened a new log
I20260926 16:37:15.120769 14240 ts_tablet_manager.cc:1397] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:15.120781 14239 raft_consensus.cc:348] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.120926 14239 raft_consensus.cc:374] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:15.120975 14239 raft_consensus.cc:729] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756a8b88bf38475e913cdb55e301ce54, State: Initialized, Role: FOLLOWER
I20260926 16:37:15.121119 14239 consensus_queue.cc:260] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.121311 14240 raft_consensus.cc:348] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.121424 14239 ts_tablet_manager.cc:1428] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:37:15.121438 14240 raft_consensus.cc:374] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:15.121475 14240 raft_consensus.cc:729] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2559e3f3d5e2471b8363521b1e707b4b, State: Initialized, Role: FOLLOWER
I20260926 16:37:15.121615 14240 consensus_queue.cc:260] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.121714 14241 tablet_bootstrap.cc:492] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: No bootstrap required, opened a new log
I20260926 16:37:15.121797 14241 ts_tablet_manager.cc:1397] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 16:37:15.121918 14240 ts_tablet_manager.cc:1428] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:37:15.122251 14241 raft_consensus.cc:348] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.122385 14241 raft_consensus.cc:374] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:15.122416 14241 raft_consensus.cc:729] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Initialized, Role: FOLLOWER
I20260926 16:37:15.122565 14241 consensus_queue.cc:260] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.122974 14241 ts_tablet_manager.cc:1428] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:37:15.160152 14245 raft_consensus.cc:482] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:15.160298 14245 raft_consensus.cc:504] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.160781 14245 leader_election.cc:288] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333)
I20260926 16:37:15.161253 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a949613dc904b6c9db218b4b50d843d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b" is_pre_election: true
I20260926 16:37:15.161248 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a949613dc904b6c9db218b4b50d843d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54" is_pre_election: true
I20260926 16:37:15.161406 13929 raft_consensus.cc:2454] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 0.
I20260926 16:37:15.161443 13789 raft_consensus.cc:2454] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 0.
I20260926 16:37:15.161788 14000 leader_election.cc:302] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:15.162019 14245 raft_consensus.cc:2781] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:15.162091 14245 raft_consensus.cc:482] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:15.162148 14245 raft_consensus.cc:3037] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:15.163153 14245 raft_consensus.cc:504] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.163515 14245 leader_election.cc:288] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 election: Requested vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333)
I20260926 16:37:15.163846 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a949613dc904b6c9db218b4b50d843d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b"
I20260926 16:37:15.163888 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a949613dc904b6c9db218b4b50d843d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54"
I20260926 16:37:15.164001 13929 raft_consensus.cc:3037] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:15.164009 13789 raft_consensus.cc:3037] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:15.165105 13789 raft_consensus.cc:2454] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 1.
I20260926 16:37:15.165113 13929 raft_consensus.cc:2454] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 1.
I20260926 16:37:15.165403 14001 leader_election.cc:302] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756a8b88bf38475e913cdb55e301ce54, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:15.165601 14245 raft_consensus.cc:2781] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:15.165829 14245 raft_consensus.cc:686] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 LEADER]: Becoming Leader. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Running, Role: LEADER
I20260926 16:37:15.165983 14245 consensus_queue.cc:237] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.167691 13632 catalog_manager.cc:5495] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a reported cstate change: term changed from 0 to 1, leader changed from <none> to b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131). New cstate: current_term: 1 leader_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } health_report { overall_health: HEALTHY } } }
I20260926 16:37:15.194618 14249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:15.191282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.11.158.131:33333)
0926 16:37:15.191532 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:15.191539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:15.191552 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:15.191592 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:15.191594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:15.191653 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:15.191733 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:15.191948 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:15.192555 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:15.193429 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:15.193776 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:15.193858 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:15.194088 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:15.194159 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:15.194214 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:15.194412 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 16:37:15.194463 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":115,"threads_started":1}
I20260926 16:37:15.201570 13789 raft_consensus.cc:1261] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Refusing update from remote peer b9a74d8eb1ae41b1accb18e2c9bfb41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:15.201665 13929 raft_consensus.cc:1261] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Refusing update from remote peer b9a74d8eb1ae41b1accb18e2c9bfb41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:15.201998 14246 consensus_queue.cc:1035] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:15.202188 14245 consensus_queue.cc:1035] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:15.237133 13631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:15.237349 13631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:15.239539 13631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Sending DeleteTablet for 3 replicas of tablet 2a949613dc904b6c9db218b4b50d843d
I20260926 16:37:15.240087 13909 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a949613dc904b6c9db218b4b50d843d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:15 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49662
I20260926 16:37:15.240137 13769 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a949613dc904b6c9db218b4b50d843d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:15 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:42076
I20260926 16:37:15.240389 14228 tablet_replica.cc:330] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: stopping tablet replica
I20260926 16:37:15.240511 14228 raft_consensus.cc:2229] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:15.240638 14228 raft_consensus.cc:2258] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:15.241530 14228 ts_tablet_manager.cc:1905] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:15.241597 14229 tablet_replica.cc:330] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: stopping tablet replica
I20260926 16:37:15.241746 14229 raft_consensus.cc:2229] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:15.241878 14229 raft_consensus.cc:2258] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:15.242244 14049 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a949613dc904b6c9db218b4b50d843d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:15 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60424
I20260926 16:37:15.242337 13631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:15.242830 13631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:15.243656 14230 tablet_replica.cc:330] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: stopping tablet replica
I20260926 16:37:15.243813 14230 raft_consensus.cc:2229] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:15.244067 14230 raft_consensus.cc:2258] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:15.244277 14228 ts_tablet_manager.cc:1918] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:15.244395 14228 log.cc:1196] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/2a949613dc904b6c9db218b4b50d843d
I20260926 16:37:15.244659 14228 ts_tablet_manager.cc:1939] T 2a949613dc904b6c9db218b4b50d843d P 756a8b88bf38475e913cdb55e301ce54: Deleting consensus metadata
I20260926 16:37:15.244776 14229 ts_tablet_manager.cc:1905] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:15.245432 13617 catalog_manager.cc:4841] TS 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807): tablet 2a949613dc904b6c9db218b4b50d843d (table test-table [id=62b375a3f03d47fe9a60743bb78cc2de]) successfully deleted
I20260926 16:37:15.245603 14230 ts_tablet_manager.cc:1905] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:15.247684 14229 ts_tablet_manager.cc:1918] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:15.247727 14230 ts_tablet_manager.cc:1918] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:15.247771 14229 log.cc:1196] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/2a949613dc904b6c9db218b4b50d843d
I20260926 16:37:15.247798 14230 log.cc:1196] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/2a949613dc904b6c9db218b4b50d843d
I20260926 16:37:15.248054 14229 ts_tablet_manager.cc:1939] T 2a949613dc904b6c9db218b4b50d843d P 2559e3f3d5e2471b8363521b1e707b4b: Deleting consensus metadata
I20260926 16:37:15.248056 14230 ts_tablet_manager.cc:1939] T 2a949613dc904b6c9db218b4b50d843d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting consensus metadata
I20260926 16:37:15.248826 13616 catalog_manager.cc:4841] TS 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333): tablet 2a949613dc904b6c9db218b4b50d843d (table test-table [id=62b375a3f03d47fe9a60743bb78cc2de]) successfully deleted
I20260926 16:37:15.249027 13619 catalog_manager.cc:4841] TS b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333): tablet 2a949613dc904b6c9db218b4b50d843d (table test-table [id=62b375a3f03d47fe9a60743bb78cc2de]) successfully deleted
I20260926 16:37:15.250370 13769 tablet_service.cc:1467] Processing CreateTablet for tablet ba49c6bebf004a7bbd01598b9603128d (DEFAULT_TABLE table=test-table [id=60247e1530b448f29ffce397edb6ecd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:15.250373 13909 tablet_service.cc:1467] Processing CreateTablet for tablet ba49c6bebf004a7bbd01598b9603128d (DEFAULT_TABLE table=test-table [id=60247e1530b448f29ffce397edb6ecd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:15.250653 13769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba49c6bebf004a7bbd01598b9603128d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:15.250655 13909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba49c6bebf004a7bbd01598b9603128d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:15.253093 14239 tablet_bootstrap.cc:492] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54: Bootstrap starting.
I20260926 16:37:15.253098 14240 tablet_bootstrap.cc:492] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: Bootstrap starting.
I20260926 16:37:15.253600 14049 tablet_service.cc:1467] Processing CreateTablet for tablet ba49c6bebf004a7bbd01598b9603128d (DEFAULT_TABLE table=test-table [id=60247e1530b448f29ffce397edb6ecd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:15.253832 14049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba49c6bebf004a7bbd01598b9603128d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:15.254014 14239 tablet_bootstrap.cc:654] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:15.254011 14240 tablet_bootstrap.cc:654] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:15.255234 14240 tablet_bootstrap.cc:492] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: No bootstrap required, opened a new log
I20260926 16:37:15.255333 14240 ts_tablet_manager.cc:1397] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 16:37:15.255473 14239 tablet_bootstrap.cc:492] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54: No bootstrap required, opened a new log
I20260926 16:37:15.255570 14239 ts_tablet_manager.cc:1397] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 16:37:15.255793 14240 raft_consensus.cc:348] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.255929 14240 raft_consensus.cc:374] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:15.255967 14240 raft_consensus.cc:729] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2559e3f3d5e2471b8363521b1e707b4b, State: Initialized, Role: FOLLOWER
I20260926 16:37:15.255975 14239 raft_consensus.cc:348] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.256042 14241 tablet_bootstrap.cc:492] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Bootstrap starting.
I20260926 16:37:15.256083 14239 raft_consensus.cc:374] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:15.256114 14239 raft_consensus.cc:729] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756a8b88bf38475e913cdb55e301ce54, State: Initialized, Role: FOLLOWER
I20260926 16:37:15.256114 14240 consensus_queue.cc:260] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.256228 14239 consensus_queue.cc:260] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.256412 14240 ts_tablet_manager.cc:1428] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:37:15.256537 14239 ts_tablet_manager.cc:1428] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:37:15.256942 14241 tablet_bootstrap.cc:654] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:15.258287 14241 tablet_bootstrap.cc:492] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: No bootstrap required, opened a new log
I20260926 16:37:15.258381 14241 ts_tablet_manager.cc:1397] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:15.258791 14241 raft_consensus.cc:348] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.258920 14241 raft_consensus.cc:374] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:15.258960 14241 raft_consensus.cc:729] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Initialized, Role: FOLLOWER
I20260926 16:37:15.259078 14241 consensus_queue.cc:260] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.259380 14241 ts_tablet_manager.cc:1428] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:15.370155 14246 raft_consensus.cc:482] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:15.370307 14246 raft_consensus.cc:504] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.370831 14246 leader_election.cc:288] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333)
I20260926 16:37:15.371315 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba49c6bebf004a7bbd01598b9603128d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b" is_pre_election: true
I20260926 16:37:15.371317 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba49c6bebf004a7bbd01598b9603128d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54" is_pre_election: true
I20260926 16:37:15.371495 13929 raft_consensus.cc:2454] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 0.
I20260926 16:37:15.371497 13789 raft_consensus.cc:2454] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 0.
I20260926 16:37:15.371830 14000 leader_election.cc:302] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:15.372030 14246 raft_consensus.cc:2781] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:15.372095 14246 raft_consensus.cc:482] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:15.372174 14246 raft_consensus.cc:3037] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:15.373369 14246 raft_consensus.cc:504] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.373767 14246 leader_election.cc:288] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 election: Requested vote from peers 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807), 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333)
I20260926 16:37:15.374137 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba49c6bebf004a7bbd01598b9603128d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2559e3f3d5e2471b8363521b1e707b4b"
I20260926 16:37:15.374162 13789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba49c6bebf004a7bbd01598b9603128d" candidate_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756a8b88bf38475e913cdb55e301ce54"
I20260926 16:37:15.374243 13929 raft_consensus.cc:3037] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:15.374302 13789 raft_consensus.cc:3037] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:15.375168 13929 raft_consensus.cc:2454] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 1.
I20260926 16:37:15.375286 13789 raft_consensus.cc:2454] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9a74d8eb1ae41b1accb18e2c9bfb41a in term 1.
I20260926 16:37:15.375479 14000 leader_election.cc:302] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2559e3f3d5e2471b8363521b1e707b4b, b9a74d8eb1ae41b1accb18e2c9bfb41a; no voters:
I20260926 16:37:15.375660 14246 raft_consensus.cc:2781] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:15.375765 14246 raft_consensus.cc:686] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 LEADER]: Becoming Leader. State: Replica: b9a74d8eb1ae41b1accb18e2c9bfb41a, State: Running, Role: LEADER
I20260926 16:37:15.375909 14246 consensus_queue.cc:237] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } }
I20260926 16:37:15.377731 13631 catalog_manager.cc:5495] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a reported cstate change: term changed from 0 to 1, leader changed from <none> to b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131). New cstate: current_term: 1 leader_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9a74d8eb1ae41b1accb18e2c9bfb41a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33333 } health_report { overall_health: HEALTHY } } }
I20260926 16:37:15.422067 13929 raft_consensus.cc:1261] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Refusing update from remote peer b9a74d8eb1ae41b1accb18e2c9bfb41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:15.422135 13789 raft_consensus.cc:1261] T ba49c6bebf004a7bbd01598b9603128d P 756a8b88bf38475e913cdb55e301ce54 [term 1 FOLLOWER]: Refusing update from remote peer b9a74d8eb1ae41b1accb18e2c9bfb41a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:15.422453 14246 consensus_queue.cc:1035] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2559e3f3d5e2471b8363521b1e707b4b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 40333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:15.422607 14245 consensus_queue.cc:1035] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [LEADER]: Connected to new peer: Peer: permanent_uuid: "756a8b88bf38475e913cdb55e301ce54" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:37:15.436133 14210 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 16:37:16.989073 13631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:16.989249 13631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:16.991401 13631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0532e93e69f64d81a517db3adb4a14c8: Sending DeleteTablet for 3 replicas of tablet ba49c6bebf004a7bbd01598b9603128d
I20260926 16:37:16.991909 13769 tablet_service.cc:1514] Processing DeleteTablet for tablet ba49c6bebf004a7bbd01598b9603128d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:16 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:42076
I20260926 16:37:16.991950 14049 tablet_service.cc:1514] Processing DeleteTablet for tablet ba49c6bebf004a7bbd01598b9603128d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:16 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60424
I20260926 16:37:16.992213 13908 tablet_service.cc:1514] Processing DeleteTablet for tablet ba49c6bebf004a7bbd01598b9603128d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:16 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49662
I20260926 16:37:16.992260 14283 tablet_replica.cc:330] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: stopping tablet replica
I20260926 16:37:16.992431 14283 raft_consensus.cc:2229] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:16.992580 14285 tablet_replica.cc:330] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: stopping tablet replica
I20260926 16:37:16.992600 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 13704
I20260926 16:37:16.993366 14285 raft_consensus.cc:2229] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:16.993484 14285 raft_consensus.cc:2258] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:16.993914 14283 raft_consensus.cc:2258] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:16.994145 14285 ts_tablet_manager.cc:1905] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:16.996412 14285 ts_tablet_manager.cc:1918] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:37:16.996542 14285 log.cc:1196] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/ba49c6bebf004a7bbd01598b9603128d
I20260926 16:37:16.996896 14285 ts_tablet_manager.cc:1939] T ba49c6bebf004a7bbd01598b9603128d P 2559e3f3d5e2471b8363521b1e707b4b: Deleting consensus metadata
I20260926 16:37:16.997520 14283 ts_tablet_manager.cc:1905] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:16.997653 13616 catalog_manager.cc:4841] TS 2559e3f3d5e2471b8363521b1e707b4b (127.11.158.130:40333): tablet ba49c6bebf004a7bbd01598b9603128d (table test-table [id=60247e1530b448f29ffce397edb6ecd9]) successfully deleted
I20260926 16:37:16.999871 14283 ts_tablet_manager.cc:1918] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:37:16.999955 14283 log.cc:1196] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/ba49c6bebf004a7bbd01598b9603128d
I20260926 16:37:17.000459 14283 ts_tablet_manager.cc:1939] T ba49c6bebf004a7bbd01598b9603128d P b9a74d8eb1ae41b1accb18e2c9bfb41a: Deleting consensus metadata
W20260926 16:37:17.000653 13617 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:43807 (error 108)
I20260926 16:37:17.000875 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 13844
I20260926 16:37:17.001344 13619 catalog_manager.cc:4841] TS b9a74d8eb1ae41b1accb18e2c9bfb41a (127.11.158.131:33333): tablet ba49c6bebf004a7bbd01598b9603128d (table test-table [id=60247e1530b448f29ffce397edb6ecd9]) successfully deleted
I20260926 16:37:17.002377 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:17.001939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43807 (local address 127.0.0.1:42106)
0926 16:37:17.002165 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:17.002242 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:43807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
W20260926 16:37:17.002558 13617 catalog_manager.cc:4584] TS 756a8b88bf38475e913cdb55e301ce54 (127.11.158.129:43807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba49c6bebf004a7bbd01598b9603128d: Network error: Client connection negotiation failed: client connection to 127.11.158.129:43807: connect: Connection refused (error 111)
I20260926 16:37:17.009198 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 13984
I20260926 16:37:17.015977 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 13600
2026-09-26T16:37:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6460 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:17 dist-test-slave-p39x krb5kdc[14290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:17 dist-test-slave-p39x krb5kdc[14290](info): set up 2 sockets
Sep 26 16:37:17 dist-test-slave-p39x krb5kdc[14290](info): commencing operation
krb5kdc: starting...
W20260926 16:37:18.057024 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:18 dist-test-slave-p39x krb5kdc[14290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:18.185220 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33527
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:18.297185 14303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:18.297624 14303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:18.297714 14303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:18.301946 14303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:18.302040 14303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:18.302078 14303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:18.302116 14303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:18.302155 14303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:18.307523 14303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:18.308703 14303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:18.309823 14303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:18.319049 14309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:18.319057 14311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:18.319065 14308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:18.319164 14303 server_base.cc:1034] running on GCE node
I20260926 16:37:18.320044 14303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:18.321280 14303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:18.322480 14303 hybrid_clock.cc:648] HybridClock initialized: now 1790440638322453 us; error 64 us; skew 500 ppm
Sep 26 16:37:18 dist-test-slave-p39x krb5kdc[14290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440638, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:18.331712 14303 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:18.332803 14303 webserver.cc:460] Webserver started at http://127.11.158.190:41343/ using document root <none> and password file <none>
I20260926 16:37:18.333369 14303 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:18.333432 14303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:18.333613 14303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:18.335742 14303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "9af449fa9a1641b58292071fc23f3f1b"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "b1d1e07f450dba08b908af97327844c2"
server_key_iv: "49e12bbf1ac9cbec39d042b053ea6485"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.336292 14303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "9af449fa9a1641b58292071fc23f3f1b"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "b1d1e07f450dba08b908af97327844c2"
server_key_iv: "49e12bbf1ac9cbec39d042b053ea6485"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.340232 14303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 16:37:18.342800 14318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:18.344273 14303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:18.344427 14303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "9af449fa9a1641b58292071fc23f3f1b"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "b1d1e07f450dba08b908af97327844c2"
server_key_iv: "49e12bbf1ac9cbec39d042b053ea6485"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.344550 14303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:18.364144 14303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:18.367640 14303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:18.367866 14303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:18.374922 14303 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:33527
I20260926 16:37:18.374984 14370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:33527 every 8 connection(s)
I20260926 16:37:18.375919 14303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:18.378752 14371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:18.382042 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 14303
I20260926 16:37:18.382153 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:18.382426 11898 external_mini_cluster.cc:1428] Setting key 9bfbca556f279022932285bd18526ee8
I20260926 16:37:18.384860 14371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b: Bootstrap starting.
I20260926 16:37:18.387058 14371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:18.387794 14371 log.cc:824] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:18.389714 14371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b: No bootstrap required, opened a new log
Sep 26 16:37:18 dist-test-slave-p39x krb5kdc[14290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:18.393299 14371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af449fa9a1641b58292071fc23f3f1b" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33527 } }
I20260926 16:37:18.393579 14371 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:18.393658 14371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af449fa9a1641b58292071fc23f3f1b, State: Initialized, Role: FOLLOWER
I20260926 16:37:18.394126 14371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af449fa9a1641b58292071fc23f3f1b" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33527 } }
I20260926 16:37:18.394277 14371 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:18.394364 14371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:18.394481 14371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:18.395350 14371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af449fa9a1641b58292071fc23f3f1b" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33527 } }
I20260926 16:37:18.395702 14371 leader_election.cc:302] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9af449fa9a1641b58292071fc23f3f1b; no voters:
I20260926 16:37:18.396018 14371 leader_election.cc:288] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:18.396137 14376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:18.396415 14376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [term 1 LEADER]: Becoming Leader. State: Replica: 9af449fa9a1641b58292071fc23f3f1b, State: Running, Role: LEADER
I20260926 16:37:18.396760 14376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af449fa9a1641b58292071fc23f3f1b" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33527 } }
I20260926 16:37:18.397295 14371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:18.398530 14377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9af449fa9a1641b58292071fc23f3f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af449fa9a1641b58292071fc23f3f1b" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33527 } } }
I20260926 16:37:18.398660 14377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:18.398897 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:18.383938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.11.158.190:33527)
0926 16:37:18.384452 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:18.384469 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:18.384523 (+ 54us) server_negotiation.cc:407] Connection header received
0926 16:37:18.385266 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:18.385286 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:18.385642 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:18.385997 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:18.386891 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.388114 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.389131 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.389651 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.389837 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:18.392601 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:18.392607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:18.392621 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:18.392650 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:18.394726 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:18.395209 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:18.395216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:18.395222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:18.395355 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:18.395674 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:18.395677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:18.395679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:18.396267 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:18.397276 (+ 1009us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:18.397437 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 16:37:18.397765 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":132,"threads_started":1}
I20260926 16:37:18.398897 14377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9af449fa9a1641b58292071fc23f3f1b. Latest consensus state: current_term: 1 leader_uuid: "9af449fa9a1641b58292071fc23f3f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af449fa9a1641b58292071fc23f3f1b" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33527 } } }
I20260926 16:37:18.399019 14377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:18.399327 14385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:18.402568 14385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:18.408414 14385 catalog_manager.cc:1347] Generated new cluster ID: dbc4a6815243474ea95820c12ac1b0a0
I20260926 16:37:18.408492 14385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:18.418201 14385 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:18.419567 14385 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:18.446790 14385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9af449fa9a1641b58292071fc23f3f1b: Generated new TSK 0
I20260926 16:37:18.447643 14385 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:18.487609 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33527
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:18.603976 14397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:18.604272 14397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:18.604322 14397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:18.608770 14397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:18.608840 14397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:18.608903 14397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:18.614499 14397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:18.615600 14397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:18.616639 14397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:18.626506 14403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:18.626616 14405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:18.626576 14397 server_base.cc:1034] running on GCE node
W20260926 16:37:18.626520 14402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:18.627236 14397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:18.627916 14397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:18.629112 14397 hybrid_clock.cc:648] HybridClock initialized: now 1790440638629090 us; error 35 us; skew 500 ppm
Sep 26 16:37:18 dist-test-slave-p39x krb5kdc[14290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440638, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:18.631968 14397 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:18.632922 14397 webserver.cc:460] Webserver started at http://127.11.158.129:43935/ using document root <none> and password file <none>
I20260926 16:37:18.633556 14397 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:18.633658 14397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:18.633891 14397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:18.635932 14397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "db7f443a4cdf4d63b73944e6694ff75f"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "1f9b8e804c34a2c4a7e54d265542cbbd"
server_key_iv: "fb580dfe0d3c5cca219e3f7db26157e0"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.636571 14397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "db7f443a4cdf4d63b73944e6694ff75f"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "1f9b8e804c34a2c4a7e54d265542cbbd"
server_key_iv: "fb580dfe0d3c5cca219e3f7db26157e0"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.640394 14397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:37:18.643034 14412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:18.644425 14397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 16:37:18.644559 14397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "db7f443a4cdf4d63b73944e6694ff75f"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "1f9b8e804c34a2c4a7e54d265542cbbd"
server_key_iv: "fb580dfe0d3c5cca219e3f7db26157e0"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.644688 14397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:18.661105 14397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:18.664746 14397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:18.664978 14397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:18.665645 14397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:18.666550 14397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:18.666626 14397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:18.666705 14397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:18.666761 14397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:18.677016 14397 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:35749
I20260926 16:37:18.677034 14525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:35749 every 8 connection(s)
I20260926 16:37:18.678041 14397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:18.684181 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 14397
I20260926 16:37:18.684358 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:18.684643 11898 external_mini_cluster.cc:1428] Setting key 35b1a4aa661e88ee8dcf670c7f68e197
Sep 26 16:37:18 dist-test-slave-p39x krb5kdc[14290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440638, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:18.694106 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:18.680015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:45065 (local address 127.11.158.190:33527)
0926 16:37:18.680166 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:18.680178 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:18.681085 (+ 907us) server_negotiation.cc:407] Connection header received
0926 16:37:18.682464 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:18.682468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:18.682551 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:18.682638 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:18.683932 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.684663 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.686119 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.686437 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.686561 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:18.689532 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:18.689538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:18.689541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:18.689566 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:18.691381 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:18.692168 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:18.692171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:18.692172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:18.692273 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:18.692797 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:18.692800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:18.692801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:18.692996 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:18.693083 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:18.693843 (+ 760us) server_negotiation.cc:299] Negotiation successful
0926 16:37:18.693963 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:37:18.695519 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:18.680410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33527 (local address 127.11.158.129:45065)
0926 16:37:18.680914 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:18.680961 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:18.682220 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:18.682847 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:18.682855 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:18.683250 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:18.683766 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:18.683781 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.684817 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.684826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:18.685946 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:18.685985 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.686611 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.686614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:18.686772 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:18.687447 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:18.687465 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:18.689286 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:18.691528 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:18.691537 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:18.691560 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:18.692030 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:18.692422 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:18.692429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:18.692434 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:18.692668 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:18.693119 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:18.693128 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:18.693519 (+ 391us) client_negotiation.cc:769] Sending connection context
0926 16:37:18.693790 (+ 271us) client_negotiation.cc:240] Negotiation successful
0926 16:37:18.694236 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":138,"threads_started":1}
I20260926 16:37:18.696991 14526 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33527
I20260926 16:37:18.697310 14526 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:18.697942 14526 heartbeater.cc:511] Master 127.11.158.190:33527 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:37:18.700075 14331 ts_manager.cc:194] Registered new tserver with Master: db7f443a4cdf4d63b73944e6694ff75f (127.11.158.129:35749)
I20260926 16:37:18.701422 14331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:45065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:18.716887 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33527
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:18.830830 14531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:18.831100 14531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:18.831144 14531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:18.835278 14531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:18.835345 14531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:18.835403 14531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:18.840842 14531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:18.841977 14531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:18.843148 14531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:18.853890 14536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:18.853950 14539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:18.853935 14537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:18.854295 14531 server_base.cc:1034] running on GCE node
I20260926 16:37:18.854732 14531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:18.855400 14531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:18.856578 14531 hybrid_clock.cc:648] HybridClock initialized: now 1790440638856538 us; error 39 us; skew 500 ppm
Sep 26 16:37:18 dist-test-slave-p39x krb5kdc[14290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440638, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:18.859376 14531 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:18.860447 14531 webserver.cc:460] Webserver started at http://127.11.158.130:33499/ using document root <none> and password file <none>
I20260926 16:37:18.861038 14531 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:18.861104 14531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:18.861327 14531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:18.863451 14531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "d0fdcc4fe2a74628b10b35082279390d"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "8cc3f4392cbb535e21a87cb3330d8a0c"
server_key_iv: "033682a830194242936d44e49e3f66a5"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.864045 14531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "d0fdcc4fe2a74628b10b35082279390d"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "8cc3f4392cbb535e21a87cb3330d8a0c"
server_key_iv: "033682a830194242936d44e49e3f66a5"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.867956 14531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:37:18.870609 14546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:18.871909 14531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:18.872015 14531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "d0fdcc4fe2a74628b10b35082279390d"
format_stamp: "Formatted at 2026-09-26 16:37:18 on dist-test-slave-p39x"
server_key: "8cc3f4392cbb535e21a87cb3330d8a0c"
server_key_iv: "033682a830194242936d44e49e3f66a5"
server_key_version: "encryptionkey@0"
I20260926 16:37:18.872165 14531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:18.901752 14531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:18.905401 14531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:18.905593 14531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:18.906184 14531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:18.907064 14531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:18.907115 14531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:18.907160 14531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:18.907177 14531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:18.916723 14531 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:39943
I20260926 16:37:18.916800 14659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:39943 every 8 connection(s)
I20260926 16:37:18.917785 14531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:18.924075 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 14531
I20260926 16:37:18.924225 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:18.924589 11898 external_mini_cluster.cc:1428] Setting key a6e9de13069179740b8256991927a026
Sep 26 16:37:18 dist-test-slave-p39x krb5kdc[14290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440638, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:18.933743 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:18.919771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:53209 (local address 127.11.158.190:33527)
0926 16:37:18.919937 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:18.919941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:18.920860 (+ 919us) server_negotiation.cc:407] Connection header received
0926 16:37:18.922342 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:18.922346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:18.922407 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:18.922537 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:18.923835 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.924586 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.926115 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.926371 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.926522 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:18.929629 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:18.929633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:18.929636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:18.929664 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:18.931266 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:18.931973 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:18.931978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:18.931980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:18.932076 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:18.932550 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:18.932555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:18.932557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:18.932761 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:18.932872 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:18.933431 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 16:37:18.933570 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:37:18.934747 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:18.920175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33527 (local address 127.11.158.130:53209)
0926 16:37:18.920688 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:18.920738 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:18.922102 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:18.922728 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:18.922736 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:18.923152 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:18.923677 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:18.923693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.924725 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.924729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:18.925978 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:18.925989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:18.926560 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:18.926564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:18.926762 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:18.927484 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:18.927502 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:18.929412 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:18.931405 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:18.931412 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:18.931433 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:18.931745 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:18.932196 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:18.932199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:18.932201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:18.932447 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:18.932871 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:18.932880 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:18.933172 (+ 292us) client_negotiation.cc:769] Sending connection context
0926 16:37:18.933375 (+ 203us) client_negotiation.cc:240] Negotiation successful
0926 16:37:18.933777 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":166,"threads_started":1}
I20260926 16:37:18.936158 14660 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:18.936594 14660 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:18.937196 14660 heartbeater.cc:511] Master 127.11.158.190:33527 requested a full tablet report, sending...
I20260926 16:37:18.938560 14331 ts_manager.cc:194] Registered new tserver with Master: d0fdcc4fe2a74628b10b35082279390d (127.11.158.130:39943)
I20260926 16:37:18.939561 14331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:53209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:18.955797 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33527
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:19.075167 14665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:19.075475 14665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:19.075556 14665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:19.079772 14665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:19.079859 14665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:19.079922 14665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:19.088290 14665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:19.090080 14665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:19.091463 14665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:19.105605 14671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:19.105605 14670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:19.106137 14665 server_base.cc:1034] running on GCE node
W20260926 16:37:19.106228 14673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:19.106859 14665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:19.107726 14665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:19.108927 14665 hybrid_clock.cc:648] HybridClock initialized: now 1790440639108847 us; error 100 us; skew 500 ppm
Sep 26 16:37:19 dist-test-slave-p39x krb5kdc[14290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440639, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:19.113207 14665 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:19.114212 14665 webserver.cc:460] Webserver started at http://127.11.158.131:36497/ using document root <none> and password file <none>
I20260926 16:37:19.114777 14665 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:19.114842 14665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:19.115021 14665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:19.117237 14665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "fd8949c0b7044c9bb2b45985193fcb6c"
format_stamp: "Formatted at 2026-09-26 16:37:19 on dist-test-slave-p39x"
server_key: "022fa792115523755fd530643fa7e9f1"
server_key_iv: "fe5d1aa3d1555537ba85a988980e159b"
server_key_version: "encryptionkey@0"
I20260926 16:37:19.117763 14665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "fd8949c0b7044c9bb2b45985193fcb6c"
format_stamp: "Formatted at 2026-09-26 16:37:19 on dist-test-slave-p39x"
server_key: "022fa792115523755fd530643fa7e9f1"
server_key_iv: "fe5d1aa3d1555537ba85a988980e159b"
server_key_version: "encryptionkey@0"
I20260926 16:37:19.121701 14665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:37:19.124585 14680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:19.126031 14665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:37:19.126184 14665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "fd8949c0b7044c9bb2b45985193fcb6c"
format_stamp: "Formatted at 2026-09-26 16:37:19 on dist-test-slave-p39x"
server_key: "022fa792115523755fd530643fa7e9f1"
server_key_iv: "fe5d1aa3d1555537ba85a988980e159b"
server_key_version: "encryptionkey@0"
I20260926 16:37:19.126313 14665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:19.144759 14665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:19.148438 14665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:19.148658 14665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:19.149374 14665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:19.150349 14665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:19.150421 14665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:19.150564 14665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:19.150601 14665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:19.161725 14665 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:45909
I20260926 16:37:19.161919 14793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:45909 every 8 connection(s)
I20260926 16:37:19.162856 14665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:19.172946 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 14665
I20260926 16:37:19.173056 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:19.173369 11898 external_mini_cluster.cc:1428] Setting key 28058db83b7f095f75ff1a4e158dc3db
Sep 26 16:37:19 dist-test-slave-p39x krb5kdc[14290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440639, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:19.178411 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:19.165272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:42311 (local address 127.11.158.190:33527)
0926 16:37:19.165429 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:19.165433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:19.166362 (+ 929us) server_negotiation.cc:407] Connection header received
0926 16:37:19.167693 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:19.167697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:19.167750 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:19.167841 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:19.169346 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:19.169897 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:19.171126 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:19.171356 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:19.171451 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:19.174405 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:19.174409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:19.174414 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:19.174445 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:19.176203 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:19.176879 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:19.176883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:19.176885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:19.176975 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:19.177323 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:19.177326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:19.177327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:19.177516 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:19.177593 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:19.178174 (+ 581us) server_negotiation.cc:299] Negotiation successful
0926 16:37:19.178290 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:37:19.179314 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:19.165638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33527 (local address 127.11.158.131:42311)
0926 16:37:19.166168 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:19.166215 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:19.167437 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:19.168062 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:19.168071 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:19.168532 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:19.169167 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:19.169183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:19.170057 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:19.170060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:19.170981 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:19.170991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:19.171504 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:19.171507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:19.171696 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:19.172503 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:19.172521 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:19.174214 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:19.176314 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:19.176319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:19.176392 (+ 73us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:19.176747 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:19.177065 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:19.177067 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:19.177069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:19.177232 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:19.177617 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:19.177623 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:19.177893 (+ 270us) client_negotiation.cc:769] Sending connection context
0926 16:37:19.178064 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 16:37:19.178354 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":148,"threads_started":1}
I20260926 16:37:19.180652 14794 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33527
I20260926 16:37:19.180930 14794 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:19.181485 14794 heartbeater.cc:511] Master 127.11.158.190:33527 requested a full tablet report, sending...
I20260926 16:37:19.182662 14331 ts_manager.cc:194] Registered new tserver with Master: fd8949c0b7044c9bb2b45985193fcb6c (127.11.158.131:45909)
I20260926 16:37:19.183248 14331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:42311
I20260926 16:37:19.188161 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 16:37:19.202507 14374 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:19.200993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.11.158.190:33527)
0926 16:37:19.201139 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:19.201143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:19.201374 (+ 231us) server_negotiation.cc:407] Connection header received
0926 16:37:19.201484 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:19.201487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:19.201532 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:19.201604 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:19.202364 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39860: BlockingRecv error: recv got EOF from 127.0.0.1:39860 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:37:19.203506 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 14397
I20260926 16:37:19.209015 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 14531
I20260926 16:37:19.214249 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 14665
I20260926 16:37:19.219604 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 14303
2026-09-26T16:37:19Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2203 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:19 dist-test-slave-p39x krb5kdc[14806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:19 dist-test-slave-p39x krb5kdc[14806](info): set up 2 sockets
Sep 26 16:37:19 dist-test-slave-p39x krb5kdc[14806](info): commencing operation
krb5kdc: starting...
W20260926 16:37:20.269155 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:20 dist-test-slave-p39x krb5kdc[14806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:20.389096 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:36929
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:20.501204 14819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:20.501530 14819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:20.501633 14819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:20.505882 14819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:20.505972 14819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:20.506006 14819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:20.506044 14819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:20.506074 14819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:20.511463 14819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:36929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:20.512634 14819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:20.513682 14819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:20.522433 14825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:20.522503 14819 server_base.cc:1034] running on GCE node
W20260926 16:37:20.522420 14824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:20.522420 14827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:20.523617 14819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:20.524824 14819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:20.526010 14819 hybrid_clock.cc:648] HybridClock initialized: now 1790440640525982 us; error 39 us; skew 500 ppm
Sep 26 16:37:20 dist-test-slave-p39x krb5kdc[14806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440640, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:20.528821 14819 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:20.529707 14819 webserver.cc:460] Webserver started at http://127.11.158.190:32781/ using document root <none> and password file <none>
I20260926 16:37:20.530212 14819 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:20.530270 14819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:20.530431 14819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:20.532498 14819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "9117f56253744481a9efd77ed4547cb5"
format_stamp: "Formatted at 2026-09-26 16:37:20 on dist-test-slave-p39x"
server_key: "e86c0f8529fe925ebc0a13cf2168688d"
server_key_iv: "1ec449651bb1bdb389860b6769eefab9"
server_key_version: "encryptionkey@0"
I20260926 16:37:20.533046 14819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "9117f56253744481a9efd77ed4547cb5"
format_stamp: "Formatted at 2026-09-26 16:37:20 on dist-test-slave-p39x"
server_key: "e86c0f8529fe925ebc0a13cf2168688d"
server_key_iv: "1ec449651bb1bdb389860b6769eefab9"
server_key_version: "encryptionkey@0"
I20260926 16:37:20.536760 14819 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:20.539059 14834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:20.540304 14819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:20.540494 14819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "9117f56253744481a9efd77ed4547cb5"
format_stamp: "Formatted at 2026-09-26 16:37:20 on dist-test-slave-p39x"
server_key: "e86c0f8529fe925ebc0a13cf2168688d"
server_key_iv: "1ec449651bb1bdb389860b6769eefab9"
server_key_version: "encryptionkey@0"
I20260926 16:37:20.540620 14819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:20.571969 14819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:20.575148 14819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:20.575400 14819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:20.582780 14819 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:36929
I20260926 16:37:20.583066 14886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:36929 every 8 connection(s)
I20260926 16:37:20.583856 14819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:20.585420 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 14819
I20260926 16:37:20.585557 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:20.585891 11898 external_mini_cluster.cc:1428] Setting key c24625af03d4b874962039e50b4242a7
I20260926 16:37:20.587769 14887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:20.595700 14887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5: Bootstrap starting.
Sep 26 16:37:20 dist-test-slave-p39x krb5kdc[14806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:20.599149 14887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:20.600265 14887 log.cc:824] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:20.602612 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:20.587347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56632 (local address 127.11.158.190:36929)
0926 16:37:20.587776 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:20.587792 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:20.587851 (+ 59us) server_negotiation.cc:407] Connection header received
0926 16:37:20.588562 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:20.588580 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:20.588919 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:20.589211 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:20.590385 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:20.591439 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:20.592802 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:20.593277 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:20.593398 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:20.596727 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:20.596736 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:20.596754 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:20.596798 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:20.599064 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:20.599620 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:20.599628 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:20.599635 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:20.599797 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:20.600087 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:20.600091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:20.600095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:20.600733 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:20.600940 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:20.601210 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 16:37:20.601577 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":138,"threads_started":1}
I20260926 16:37:20.602932 14887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5: No bootstrap required, opened a new log
I20260926 16:37:20.605724 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:37:20.607705 14887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9117f56253744481a9efd77ed4547cb5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36929 } }
I20260926 16:37:20.608064 14887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:20.608155 14887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9117f56253744481a9efd77ed4547cb5, State: Initialized, Role: FOLLOWER
I20260926 16:37:20.608775 14887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9117f56253744481a9efd77ed4547cb5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36929 } }
I20260926 16:37:20.608985 14887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:20.609072 14887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:20.609205 14887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:20.610903 14887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9117f56253744481a9efd77ed4547cb5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36929 } }
I20260926 16:37:20.611351 14887 leader_election.cc:302] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9117f56253744481a9efd77ed4547cb5; no voters:
I20260926 16:37:20.611809 14887 leader_election.cc:288] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:20.611939 14893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:20.612178 14893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [term 1 LEADER]: Becoming Leader. State: Replica: 9117f56253744481a9efd77ed4547cb5, State: Running, Role: LEADER
I20260926 16:37:20.612627 14893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9117f56253744481a9efd77ed4547cb5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36929 } }
I20260926 16:37:20.613485 14887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:20.613893 14893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9117f56253744481a9efd77ed4547cb5. Latest consensus state: current_term: 1 leader_uuid: "9117f56253744481a9efd77ed4547cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9117f56253744481a9efd77ed4547cb5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36929 } } }
I20260926 16:37:20.614107 14893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:20.614301 14894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9117f56253744481a9efd77ed4547cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9117f56253744481a9efd77ed4547cb5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36929 } } }
I20260926 16:37:20.614393 14894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:20.614619 14898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:20.617883 14898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:20.624541 14898 catalog_manager.cc:1347] Generated new cluster ID: 238c55cc271243f4a196127ede907d9e
I20260926 16:37:20.624626 14898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:20.627622 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:20.621062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.11.158.190:36929)
0926 16:37:20.621215 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:20.621220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:20.622711 (+ 1491us) server_negotiation.cc:407] Connection header received
0926 16:37:20.622873 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:20.622876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:20.622917 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:20.622987 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:20.623773 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:20.624559 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:20.625468 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:20.625734 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:20.625844 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:20.626728 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:20.626732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:20.626734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:20.627155 (+ 421us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:20.627185 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:20.627201 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:20.627303 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:20.627414 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 16:37:20.627481 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:37:20.675868 14898 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:20.677021 14898 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:20.683669 14898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9117f56253744481a9efd77ed4547cb5: Generated new TSK 0
I20260926 16:37:20.684922 14898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:37:20.700309 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 14819
2026-09-26T16:37:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1478 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:20 dist-test-slave-p39x krb5kdc[14920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:20 dist-test-slave-p39x krb5kdc[14920](info): set up 2 sockets
Sep 26 16:37:20 dist-test-slave-p39x krb5kdc[14920](info): commencing operation
krb5kdc: starting...
W20260926 16:37:21.728997 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:21 dist-test-slave-p39x krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:21.860325 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33001
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:21.974778 14933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:21.975083 14933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:21.975126 14933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:21.979301 14933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:21.979384 14933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:21.979408 14933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:21.979424 14933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:21.979440 14933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:21.984876 14933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:21.986060 14933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:21.987104 14933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:21.995898 14941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:21.995889 14938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:21.995899 14939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:21.995941 14933 server_base.cc:1034] running on GCE node
I20260926 16:37:21.996917 14933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:21.998167 14933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:21.999431 14933 hybrid_clock.cc:648] HybridClock initialized: now 1790440641999415 us; error 38 us; skew 500 ppm
Sep 26 16:37:21 dist-test-slave-p39x krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440641, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:22.002455 14933 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:22.003441 14933 webserver.cc:460] Webserver started at http://127.11.158.190:37487/ using document root <none> and password file <none>
I20260926 16:37:22.004055 14933 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:22.004153 14933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:22.004454 14933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:22.006820 14933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "271ca552275c4dee8e94c1fc5ca320ff"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "161115ea8567ae3377cab0218241e6d4"
server_key_iv: "34de26623f1a75896a4200ca0225463e"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.007433 14933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "271ca552275c4dee8e94c1fc5ca320ff"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "161115ea8567ae3377cab0218241e6d4"
server_key_iv: "34de26623f1a75896a4200ca0225463e"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.011551 14933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 16:37:22.014315 14948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.015743 14933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 16:37:22.015906 14933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "271ca552275c4dee8e94c1fc5ca320ff"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "161115ea8567ae3377cab0218241e6d4"
server_key_iv: "34de26623f1a75896a4200ca0225463e"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.016031 14933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:22.029457 14933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:22.032748 14933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:22.032969 14933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:22.040235 14933 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:33001
I20260926 16:37:22.040571 15000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:33001 every 8 connection(s)
I20260926 16:37:22.041301 14933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:22.044092 15001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:22.047022 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 14933
I20260926 16:37:22.047163 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:22.047480 11898 external_mini_cluster.cc:1428] Setting key 3c3b3fc0af4d84195de09a0ba86bccfe
I20260926 16:37:22.050305 15001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff: Bootstrap starting.
I20260926 16:37:22.053047 15001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:22.053970 15001 log.cc:824] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:22.056501 15001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff: No bootstrap required, opened a new log
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:22.059975 15001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271ca552275c4dee8e94c1fc5ca320ff" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33001 } }
I20260926 16:37:22.060240 15001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:22.060323 15001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271ca552275c4dee8e94c1fc5ca320ff, State: Initialized, Role: FOLLOWER
I20260926 16:37:22.060876 15001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271ca552275c4dee8e94c1fc5ca320ff" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33001 } }
I20260926 16:37:22.061005 15001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:22.061089 15001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:22.061214 15001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:22.062345 15001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271ca552275c4dee8e94c1fc5ca320ff" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33001 } }
I20260926 16:37:22.062703 15001 leader_election.cc:302] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271ca552275c4dee8e94c1fc5ca320ff; no voters:
I20260926 16:37:22.062949 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.048969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.11.158.190:33001)
0926 16:37:22.049432 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:22.049448 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:22.049491 (+ 43us) server_negotiation.cc:407] Connection header received
0926 16:37:22.050120 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:22.050137 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:22.050541 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:22.050829 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:22.051786 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.052897 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.054156 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.054654 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.054783 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.057536 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:22.057541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:22.057558 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:22.057589 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:22.059730 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.060200 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.060207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.060213 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.060438 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.060752 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.060756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.060757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.061295 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:22.061481 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:22.061795 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 16:37:22.062097 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":151,"threads_started":1}
I20260926 16:37:22.063071 15001 leader_election.cc:288] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:22.063151 15006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:22.063337 15006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [term 1 LEADER]: Becoming Leader. State: Replica: 271ca552275c4dee8e94c1fc5ca320ff, State: Running, Role: LEADER
I20260926 16:37:22.063637 15006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271ca552275c4dee8e94c1fc5ca320ff" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33001 } }
I20260926 16:37:22.064419 15001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:22.065337 15007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "271ca552275c4dee8e94c1fc5ca320ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271ca552275c4dee8e94c1fc5ca320ff" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33001 } } }
I20260926 16:37:22.065713 15007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:22.065618 15008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271ca552275c4dee8e94c1fc5ca320ff. Latest consensus state: current_term: 1 leader_uuid: "271ca552275c4dee8e94c1fc5ca320ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271ca552275c4dee8e94c1fc5ca320ff" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33001 } } }
I20260926 16:37:22.066033 15008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:22.066190 15015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:22.068900 15015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:22.074801 15015 catalog_manager.cc:1347] Generated new cluster ID: 2d2224aacc1940a1a3fdbadd74758318
I20260926 16:37:22.074878 15015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:22.090619 15015 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:22.091542 15015 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:22.098625 15015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 271ca552275c4dee8e94c1fc5ca320ff: Generated new TSK 0
I20260926 16:37:22.099319 15015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:22.135648 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33001
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:22.274561 15027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:22.274893 15027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:22.274984 15027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:22.279137 15027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:22.279239 15027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:22.279340 15027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:22.284737 15027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:22.285840 15027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:22.286942 15027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:22.296928 15032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:22.296972 15033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:22.297149 15027 server_base.cc:1034] running on GCE node
W20260926 16:37:22.296882 15035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:22.297755 15027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:22.298420 15027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:22.299642 15027 hybrid_clock.cc:648] HybridClock initialized: now 1790440642299626 us; error 34 us; skew 500 ppm
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:22.302599 15027 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:22.303503 15027 webserver.cc:460] Webserver started at http://127.11.158.129:34015/ using document root <none> and password file <none>
I20260926 16:37:22.304076 15027 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:22.304163 15027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:22.304445 15027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:22.306474 15027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "43c02c72595043d197178243a5202487"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "d2015f60c93327a2643831a53fa7f3a8"
server_key_iv: "6ec400552f8c52d5980a98ab23351a9f"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.307037 15027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "43c02c72595043d197178243a5202487"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "d2015f60c93327a2643831a53fa7f3a8"
server_key_iv: "6ec400552f8c52d5980a98ab23351a9f"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.310958 15027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:37:22.313580 15042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.314914 15027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:22.315079 15027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "43c02c72595043d197178243a5202487"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "d2015f60c93327a2643831a53fa7f3a8"
server_key_iv: "6ec400552f8c52d5980a98ab23351a9f"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.315205 15027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:22.334877 15027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:22.338455 15027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:22.338670 15027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:22.339295 15027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:22.340147 15027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:22.340202 15027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.340286 15027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:22.340325 15027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.348989 15027 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:42459
I20260926 16:37:22.349054 15155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:42459 every 8 connection(s)
I20260926 16:37:22.349942 15027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:22.353219 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15027
I20260926 16:37:22.353353 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:22.353672 11898 external_mini_cluster.cc:1428] Setting key f82b754ae3190d884e121b8f158dd982
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:37:22.365967 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.351894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:40613 (local address 127.11.158.190:33001)
0926 16:37:22.352047 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:22.352052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:22.353077 (+ 1025us) server_negotiation.cc:407] Connection header received
0926 16:37:22.354887 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:22.354890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:22.354935 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:22.355007 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:22.356398 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.357183 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.358809 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.359241 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.359377 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.361976 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:22.361980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:22.361985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:22.362019 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:22.363671 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.364220 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.364225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.364227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.364378 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.364759 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.364763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.364764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.365016 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:22.365095 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:22.365699 (+ 604us) server_negotiation.cc:299] Negotiation successful
0926 16:37:22.365809 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:37:22.367091 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.352320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33001 (local address 127.11.158.129:40613)
0926 16:37:22.352882 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:22.352964 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:22.354630 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:22.355264 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:22.355277 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:22.355697 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:22.356196 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:22.356209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.357339 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.357344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:22.358650 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:22.358659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.359406 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.359408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:22.359638 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.360286 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:22.360303 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:22.361794 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:22.363829 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:22.363834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:22.363847 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:22.364097 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:22.364470 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:22.364472 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:22.364474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:22.364650 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:22.365099 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:22.365104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:22.365410 (+ 306us) client_negotiation.cc:769] Sending connection context
0926 16:37:22.365614 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 16:37:22.365999 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":125,"threads_started":1}
I20260926 16:37:22.368543 15156 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33001
I20260926 16:37:22.368839 15156 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:22.369395 15156 heartbeater.cc:511] Master 127.11.158.190:33001 requested a full tablet report, sending...
I20260926 16:37:22.370824 14963 ts_manager.cc:194] Registered new tserver with Master: 43c02c72595043d197178243a5202487 (127.11.158.129:42459)
I20260926 16:37:22.371953 14963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:40613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:22.384449 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33001
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:22.496354 15161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:22.496620 15161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:22.496663 15161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:22.500730 15161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:22.500795 15161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:22.500855 15161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:22.506073 15161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:22.507136 15161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:22.508183 15161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:22.518169 15169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:22.518141 15167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:22.518124 15166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:22.518504 15161 server_base.cc:1034] running on GCE node
I20260926 16:37:22.519037 15161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:22.519866 15161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:22.521054 15161 hybrid_clock.cc:648] HybridClock initialized: now 1790440642521036 us; error 30 us; skew 500 ppm
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:22.529964 15161 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:22.530952 15161 webserver.cc:460] Webserver started at http://127.11.158.130:38887/ using document root <none> and password file <none>
I20260926 16:37:22.531495 15161 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:22.531555 15161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:22.531726 15161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:22.533844 15161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "7d6e89f80bb24e37a2834647501f855c"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "c3ef9f3d4759666cc47b8f0bf83864b3"
server_key_iv: "388282bdf5a2da3f4342b2255852d5b0"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.534364 15161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "7d6e89f80bb24e37a2834647501f855c"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "c3ef9f3d4759666cc47b8f0bf83864b3"
server_key_iv: "388282bdf5a2da3f4342b2255852d5b0"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.538290 15161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:37:22.540717 15176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.541953 15161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:22.542080 15161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "7d6e89f80bb24e37a2834647501f855c"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "c3ef9f3d4759666cc47b8f0bf83864b3"
server_key_iv: "388282bdf5a2da3f4342b2255852d5b0"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.542198 15161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:22.553092 15161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:22.556397 15161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:22.556588 15161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:22.557164 15161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:22.557967 15161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:22.558014 15161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.558081 15161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:22.558117 15161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.567427 15161 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:41329
I20260926 16:37:22.567445 15289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:41329 every 8 connection(s)
I20260926 16:37:22.568380 15161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:22.570979 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15161
I20260926 16:37:22.571123 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:22.571437 11898 external_mini_cluster.cc:1428] Setting key e9c5b5176d734c46ee51a521d2124e99
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:22.585013 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.570136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:45425 (local address 127.11.158.190:33001)
0926 16:37:22.570265 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:22.570269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:22.571173 (+ 904us) server_negotiation.cc:407] Connection header received
0926 16:37:22.572949 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:22.572952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:22.573014 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:22.573094 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:22.574260 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.574924 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.576759 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.577045 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.577162 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.580089 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:22.580094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:22.580098 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:22.580132 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:22.581770 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.582646 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.582663 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.582672 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.582818 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.583244 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.583247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.583249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.583490 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:22.583609 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:22.584629 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0926 16:37:22.584788 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:37:22.587375 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.570479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33001 (local address 127.11.158.130:45425)
0926 16:37:22.570985 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:22.571063 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:22.572726 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:22.573278 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:22.573285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:22.573637 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:22.574116 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:22.574130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.575055 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.575060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:22.576621 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:22.576635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.577252 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.577259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:22.577476 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.578124 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:22.578152 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:22.579910 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:22.581914 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:22.581920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:22.581944 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:22.582378 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:22.582944 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:22.582946 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:22.582948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:22.583127 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:22.583617 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:22.583624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:22.584161 (+ 537us) client_negotiation.cc:769] Sending connection context
0926 16:37:22.585305 (+ 1144us) client_negotiation.cc:240] Negotiation successful
0926 16:37:22.585866 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":110,"threads_started":1}
I20260926 16:37:22.588760 15290 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33001
I20260926 16:37:22.589027 15290 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:22.589779 15290 heartbeater.cc:511] Master 127.11.158.190:33001 requested a full tablet report, sending...
I20260926 16:37:22.591039 14963 ts_manager.cc:194] Registered new tserver with Master: 7d6e89f80bb24e37a2834647501f855c (127.11.158.130:41329)
I20260926 16:37:22.591601 14963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:45425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:22.607095 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33001
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:22.731115 15295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:22.731407 15295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:22.731451 15295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:22.735776 15295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:22.735850 15295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:22.735939 15295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:22.741580 15295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:22.742748 15295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:22.743829 15295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:37:22.754011 15295 server_base.cc:1034] running on GCE node
W20260926 16:37:22.754001 15301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:22.753961 15303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:22.753961 15300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:22.754714 15295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:22.755364 15295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:22.756547 15295 hybrid_clock.cc:648] HybridClock initialized: now 1790440642756515 us; error 48 us; skew 500 ppm
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:22.759392 15295 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:22.760426 15295 webserver.cc:460] Webserver started at http://127.11.158.131:37893/ using document root <none> and password file <none>
I20260926 16:37:22.761020 15295 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:22.761111 15295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:22.761354 15295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:22.763511 15295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "5771b1ab06b449f9814013e548404afc"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "e759fdf8a86d6338f58f113b79617709"
server_key_iv: "ce53adec33120d6ac30fc1ba3d01253e"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.764127 15295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "5771b1ab06b449f9814013e548404afc"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "e759fdf8a86d6338f58f113b79617709"
server_key_iv: "ce53adec33120d6ac30fc1ba3d01253e"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.768329 15295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 16:37:22.771164 15310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.772557 15295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:22.772713 15295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "5771b1ab06b449f9814013e548404afc"
format_stamp: "Formatted at 2026-09-26 16:37:22 on dist-test-slave-p39x"
server_key: "e759fdf8a86d6338f58f113b79617709"
server_key_iv: "ce53adec33120d6ac30fc1ba3d01253e"
server_key_version: "encryptionkey@0"
I20260926 16:37:22.772840 15295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:22.789693 15295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:22.792950 15295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:22.793159 15295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:22.793767 15295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:22.794648 15295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:22.794727 15295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.794800 15295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:22.794843 15295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:22.804116 15295 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:41017
I20260926 16:37:22.804122 15423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:41017 every 8 connection(s)
I20260926 16:37:22.805040 15295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:22.814483 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15295
I20260926 16:37:22.814586 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:22.814814 11898 external_mini_cluster.cc:1428] Setting key cd73d7d282474912dfa53b11534b5d23
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:22.819609 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.806889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:59105 (local address 127.11.158.190:33001)
0926 16:37:22.807067 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:22.807070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:22.807837 (+ 767us) server_negotiation.cc:407] Connection header received
0926 16:37:22.809389 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:22.809392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:22.809461 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:22.809563 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:22.810830 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.811431 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.812757 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.812987 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.813108 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.815772 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:22.815776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:22.815779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:22.815812 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:22.817451 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.817939 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.817942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.817943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.818018 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.818362 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.818365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.818366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.818546 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:22.818623 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:22.819284 (+ 661us) server_negotiation.cc:299] Negotiation successful
0926 16:37:22.819412 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:37:22.820593 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.807201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33001 (local address 127.11.158.131:59105)
0926 16:37:22.807670 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:22.807716 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:22.809158 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:22.809755 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:22.809762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:22.810165 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:22.810673 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:22.810687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.811578 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.811587 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:22.812629 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:22.812646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.813151 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.813156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:22.813337 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.813959 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:22.813976 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:22.815589 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:22.817552 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:22.817557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:22.817571 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:22.817853 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:22.818094 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:22.818096 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:22.818098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:22.818275 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:22.818655 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:22.818661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:22.818958 (+ 297us) client_negotiation.cc:769] Sending connection context
0926 16:37:22.819212 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 16:37:22.819568 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":108,"threads_started":1}
I20260926 16:37:22.821727 15424 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33001
I20260926 16:37:22.821946 15424 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:22.822396 15424 heartbeater.cc:511] Master 127.11.158.190:33001 requested a full tablet report, sending...
I20260926 16:37:22.823410 14963 ts_manager.cc:194] Registered new tserver with Master: 5771b1ab06b449f9814013e548404afc (127.11.158.131:41017)
I20260926 16:37:22.823997 14963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:59105
I20260926 16:37:22.829320 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[14920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440642, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:22.856192 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:22.846399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.11.158.190:33001)
0926 16:37:22.846588 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:22.846593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:22.846807 (+ 214us) server_negotiation.cc:407] Connection header received
0926 16:37:22.846987 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:22.846991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:22.847053 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:22.847143 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:22.847885 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.848500 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.849349 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:22.849589 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:22.849705 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:22.852511 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:22.852515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:22.852517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:22.852547 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:22.854190 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.854740 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.854746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.854748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.854848 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:22.855172 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:22.855176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:22.855178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:22.855430 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:22.855582 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:22.855862 (+ 280us) server_negotiation.cc:299] Negotiation successful
0926 16:37:22.856003 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260926 16:37:22.856809 14963 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53708
I20260926 16:37:22.857963 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15027
I20260926 16:37:22.864445 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15161
I20260926 16:37:22.870553 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15295
I20260926 16:37:22.875880 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 14933
2026-09-26T16:37:22Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2177 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[15436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[15436](info): set up 2 sockets
Sep 26 16:37:22 dist-test-slave-p39x krb5kdc[15436](info): commencing operation
krb5kdc: starting...
W20260926 16:37:23.913164 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:23 dist-test-slave-p39x krb5kdc[15436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:24.040190 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:41527
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:24.157622 15449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:24.157938 15449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:24.158033 15449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:24.162168 15449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:24.162276 15449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:24.162312 15449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:24.162352 15449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:24.162389 15449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:24.168256 15449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:41527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:41527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:24.169492 15449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:24.170564 15449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:24.179333 15457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:24.179428 15449 server_base.cc:1034] running on GCE node
W20260926 16:37:24.179438 15454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:24.179346 15455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:24.180172 15449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:24.181365 15449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:24.182564 15449 hybrid_clock.cc:648] HybridClock initialized: now 1790440644182531 us; error 59 us; skew 500 ppm
Sep 26 16:37:24 dist-test-slave-p39x krb5kdc[15436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440644, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:24.191525 15449 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:24.192643 15449 webserver.cc:460] Webserver started at http://127.11.158.190:36153/ using document root <none> and password file <none>
I20260926 16:37:24.193260 15449 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:24.193358 15449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:24.193600 15449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:24.195746 15449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "d83d76f016b747dc9dbcfd60ff19f31a"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "0c122af69de9349627495e62c71a1f15"
server_key_iv: "93b67119dcde4fd7ddbe680da8db4996"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.196326 15449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "d83d76f016b747dc9dbcfd60ff19f31a"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "0c122af69de9349627495e62c71a1f15"
server_key_iv: "93b67119dcde4fd7ddbe680da8db4996"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.200172 15449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:37:24.202649 15464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:24.203912 15449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:24.204046 15449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "d83d76f016b747dc9dbcfd60ff19f31a"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "0c122af69de9349627495e62c71a1f15"
server_key_iv: "93b67119dcde4fd7ddbe680da8db4996"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.204154 15449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:24.215167 15449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:24.218529 15449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:24.218766 15449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:24.225898 15449 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:41527
I20260926 16:37:24.226029 15516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:41527 every 8 connection(s)
I20260926 16:37:24.227159 15449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:24.230628 15517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:24.236793 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15449
I20260926 16:37:24.236896 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:24.237177 11898 external_mini_cluster.cc:1428] Setting key 263800dcb7c31ebc0d637448ed30353f
I20260926 16:37:24.237360 15517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a: Bootstrap starting.
I20260926 16:37:24.239903 15517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:24.240728 15517 log.cc:824] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:24.242604 15517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a: No bootstrap required, opened a new log
Sep 26 16:37:24 dist-test-slave-p39x krb5kdc[15436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:24.246189 15517 raft_consensus.cc:348] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41527 } }
I20260926 16:37:24.246450 15517 raft_consensus.cc:374] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:24.246529 15517 raft_consensus.cc:729] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83d76f016b747dc9dbcfd60ff19f31a, State: Initialized, Role: FOLLOWER
I20260926 16:37:24.247017 15517 consensus_queue.cc:260] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41527 } }
I20260926 16:37:24.247181 15517 raft_consensus.cc:388] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:24.247268 15517 raft_consensus.cc:482] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:24.247372 15517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:24.248287 15517 raft_consensus.cc:504] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41527 } }
I20260926 16:37:24.248659 15517 leader_election.cc:302] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d83d76f016b747dc9dbcfd60ff19f31a; no voters:
I20260926 16:37:24.249015 15517 leader_election.cc:288] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:24.249089 15522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:24.249286 15522 raft_consensus.cc:686] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [term 1 LEADER]: Becoming Leader. State: Replica: d83d76f016b747dc9dbcfd60ff19f31a, State: Running, Role: LEADER
I20260926 16:37:24.249585 15522 consensus_queue.cc:237] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41527 } }
I20260926 16:37:24.250355 15517 sys_catalog.cc:564] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:24.250787 15522 sys_catalog.cc:455] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d83d76f016b747dc9dbcfd60ff19f31a. Latest consensus state: current_term: 1 leader_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41527 } } }
I20260926 16:37:24.250909 15522 sys_catalog.cc:458] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:24.251104 15522 sys_catalog.cc:455] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83d76f016b747dc9dbcfd60ff19f31a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41527 } } }
I20260926 16:37:24.251202 15522 sys_catalog.cc:458] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:24.251469 15520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:24.238522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.11.158.190:41527)
0926 16:37:24.239011 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:24.239026 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:24.239061 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:37:24.239706 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:24.239724 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:24.240098 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:24.240491 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:24.241303 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.242277 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.243213 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.243684 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.243858 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:24.246349 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:24.246354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:24.246368 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:24.246397 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:24.248257 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:24.248744 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:24.248750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:24.248756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:24.248898 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:24.249247 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:24.249257 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:24.249259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:24.249753 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:24.249942 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:24.250219 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 16:37:24.250538 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":157,"threads_started":1}
I20260926 16:37:24.251493 15526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:24.255556 15526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:24.261452 15526 catalog_manager.cc:1347] Generated new cluster ID: a27830af780f434491f7855475c03e3f
I20260926 16:37:24.261538 15526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:24.276328 15526 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:24.277688 15526 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:24.286795 15526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d83d76f016b747dc9dbcfd60ff19f31a: Generated new TSK 0
I20260926 16:37:24.287689 15526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:24.325321 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:41527
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:24.436879 15543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:24.437157 15543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:24.437232 15543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:24.441232 15543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:24.441304 15543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:24.441390 15543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:24.446430 15543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:24.447394 15543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:24.448410 15543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:24.457697 15551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:24.457736 15548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:24.457882 15543 server_base.cc:1034] running on GCE node
W20260926 16:37:24.457717 15549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:24.458397 15543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:24.459009 15543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:24.460201 15543 hybrid_clock.cc:648] HybridClock initialized: now 1790440644460184 us; error 30 us; skew 500 ppm
Sep 26 16:37:24 dist-test-slave-p39x krb5kdc[15436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440644, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:24.462869 15543 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:24.463692 15543 webserver.cc:460] Webserver started at http://127.11.158.129:35169/ using document root <none> and password file <none>
I20260926 16:37:24.464233 15543 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:24.464324 15543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:24.464545 15543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:24.466465 15543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "2d7132ea76734e43986df6dd8da962b9"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "f537cf7b37dbe398198e9338e633353a"
server_key_iv: "b2b5a6d3d0eadcf5e84649ae714f443e"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.467010 15543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "2d7132ea76734e43986df6dd8da962b9"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "f537cf7b37dbe398198e9338e633353a"
server_key_iv: "b2b5a6d3d0eadcf5e84649ae714f443e"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.470700 15543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 16:37:24.473019 15558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:24.474165 15543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:37:24.474291 15543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "2d7132ea76734e43986df6dd8da962b9"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "f537cf7b37dbe398198e9338e633353a"
server_key_iv: "b2b5a6d3d0eadcf5e84649ae714f443e"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.474395 15543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:24.492739 15543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:24.495400 15543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:24.495594 15543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:24.496160 15543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:24.496986 15543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:24.497056 15543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:24.497133 15543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:24.497174 15543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:24.506297 15543 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:44609
I20260926 16:37:24.506319 15671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:44609 every 8 connection(s)
I20260926 16:37:24.507167 15543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:24.511879 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15543
I20260926 16:37:24.511977 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:24.512401 11898 external_mini_cluster.cc:1428] Setting key df1de5511df1c9b233a4b912cc191f10
Sep 26 16:37:24 dist-test-slave-p39x krb5kdc[15436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440644, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:24.521446 15520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:24.509015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:54861 (local address 127.11.158.190:41527)
0926 16:37:24.509154 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:24.509159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:24.510104 (+ 945us) server_negotiation.cc:407] Connection header received
0926 16:37:24.511411 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:24.511414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:24.511461 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:24.511535 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:24.512983 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.513623 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.514768 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.515191 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.515303 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:24.517874 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:24.517878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:24.517880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:24.517904 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:24.519348 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:24.519921 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:24.519925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:24.519927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:24.520012 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:24.520379 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:24.520384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:24.520389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:24.520572 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:24.520681 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:24.521156 (+ 475us) server_negotiation.cc:299] Negotiation successful
0926 16:37:24.521275 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:37:24.522255 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:24.509473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:41527 (local address 127.11.158.129:54861)
0926 16:37:24.509958 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:24.510000 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:24.511221 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:24.511691 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:24.511698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:24.512150 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:24.512796 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:24.512815 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.513750 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.513753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:24.514636 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:24.514645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.515355 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.515360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:24.515573 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:24.516200 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:24.516224 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:24.517704 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:24.519481 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:24.519486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:24.519501 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:24.519814 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:24.520112 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:24.520115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:24.520117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:24.520265 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:24.520679 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:24.520685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:24.520919 (+ 234us) client_negotiation.cc:769] Sending connection context
0926 16:37:24.521101 (+ 182us) client_negotiation.cc:240] Negotiation successful
0926 16:37:24.521414 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":120,"threads_started":1}
I20260926 16:37:24.523413 15672 heartbeater.cc:347] Connected to a master server at 127.11.158.190:41527
I20260926 16:37:24.523680 15672 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:37:24.524282 15672 heartbeater.cc:511] Master 127.11.158.190:41527 requested a full tablet report, sending...
I20260926 16:37:24.525792 15481 ts_manager.cc:194] Registered new tserver with Master: 2d7132ea76734e43986df6dd8da962b9 (127.11.158.129:44609)
I20260926 16:37:24.526876 15481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:54861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:24.538980 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:41527
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:24.652878 15677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:24.653172 15677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:24.653246 15677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:24.657326 15677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:24.657430 15677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:24.657528 15677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:24.662786 15677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:24.663892 15677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:24.664958 15677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:24.674970 15683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:24.674957 15685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:24.675038 15677 server_base.cc:1034] running on GCE node
W20260926 16:37:24.674965 15682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:24.675673 15677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:24.676416 15677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:24.679708 15677 hybrid_clock.cc:648] HybridClock initialized: now 1790440644679405 us; error 305 us; skew 500 ppm
Sep 26 16:37:24 dist-test-slave-p39x krb5kdc[15436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440644, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:24.683327 15677 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:24.684319 15677 webserver.cc:460] Webserver started at http://127.11.158.130:37657/ using document root <none> and password file <none>
I20260926 16:37:24.684955 15677 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:24.685050 15677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:24.685304 15677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:24.687328 15677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "68fa56417cce4131aeee5a26bddaefff"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "43d20a94fe91f8dc180008720f092499"
server_key_iv: "f7f9ac371e2e585f999af599a158a74d"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.687915 15677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "68fa56417cce4131aeee5a26bddaefff"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "43d20a94fe91f8dc180008720f092499"
server_key_iv: "f7f9ac371e2e585f999af599a158a74d"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.691622 15677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260926 16:37:24.694229 15692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:24.695622 15677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:37:24.695773 15677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "68fa56417cce4131aeee5a26bddaefff"
format_stamp: "Formatted at 2026-09-26 16:37:24 on dist-test-slave-p39x"
server_key: "43d20a94fe91f8dc180008720f092499"
server_key_iv: "f7f9ac371e2e585f999af599a158a74d"
server_key_version: "encryptionkey@0"
I20260926 16:37:24.695897 15677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:24.712144 15677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:24.715389 15677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:24.715602 15677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:24.716209 15677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:24.717130 15677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:24.717203 15677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:24.717274 15677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:24.717329 15677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:24.726938 15677 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:34335
I20260926 16:37:24.726962 15805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:34335 every 8 connection(s)
I20260926 16:37:24.727867 15677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:24.734935 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15677
I20260926 16:37:24.735097 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:24.735379 11898 external_mini_cluster.cc:1428] Setting key 69f820bed4bbd2f6322a225825230eb3
Sep 26 16:37:24 dist-test-slave-p39x krb5kdc[15436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440644, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:24.742545 15520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:24.729779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:36211 (local address 127.11.158.190:41527)
0926 16:37:24.729978 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:24.729982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:24.730850 (+ 868us) server_negotiation.cc:407] Connection header received
0926 16:37:24.732308 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:24.732312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:24.732401 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:24.732507 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:24.733762 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.734338 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.735601 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.735879 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.735991 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:24.738888 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:24.738891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:24.738893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:24.738918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:24.740399 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:24.740965 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:24.740969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:24.740971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:24.741053 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:24.741387 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:24.741392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:24.741394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:24.741656 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:24.741774 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:24.742269 (+ 495us) server_negotiation.cc:299] Negotiation successful
0926 16:37:24.742387 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 16:37:24.743356 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:24.730194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:41527 (local address 127.11.158.130:36211)
0926 16:37:24.730689 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:24.730734 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:24.732074 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:24.732704 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:24.732711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:24.733107 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:24.733586 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:24.733600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.734480 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.734483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:24.735472 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:24.735484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:24.736047 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:24.736051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:24.736245 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:24.736918 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:24.736939 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:24.738692 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:24.740508 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:24.740516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:24.740531 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:24.740861 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:24.741151 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:24.741155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:24.741157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:24.741291 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:24.741772 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:24.741780 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:24.742030 (+ 250us) client_negotiation.cc:769] Sending connection context
0926 16:37:24.742228 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 16:37:24.742548 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":109,"threads_started":1}
I20260926 16:37:24.744575 15806 heartbeater.cc:347] Connected to a master server at 127.11.158.190:41527
I20260926 16:37:24.744794 15806 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:24.745249 15806 heartbeater.cc:511] Master 127.11.158.190:41527 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:37:24.747225 15481 ts_manager.cc:194] Registered new tserver with Master: 68fa56417cce4131aeee5a26bddaefff (127.11.158.130:34335)
I20260926 16:37:24.747761 15481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:36211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:25.529254 15672 heartbeater.cc:503] Master 127.11.158.190:41527 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
W20260926 16:37:25.728189 11898 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.158.131: real 0.991s user 0.002s sys 0.002s
I20260926 16:37:25.728950 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:41527
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:37:25.749796 15806 heartbeater.cc:503] Master 127.11.158.190:41527 was elected leader, sending a full tablet report...
W20260926 16:37:25.928642 15811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:25.929037 15811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:25.929148 15811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:25.934428 15811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:25.934556 15811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:25.934666 15811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:25.940820 15811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:25.942052 15811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:25.943174 15811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:25.953877 15816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:25.953877 15819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:25.954067 15811 server_base.cc:1034] running on GCE node
W20260926 16:37:25.953936 15817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:25.954671 15811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:25.955415 15811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:25.956615 15811 hybrid_clock.cc:648] HybridClock initialized: now 1790440645956575 us; error 67 us; skew 500 ppm
Sep 26 16:37:25 dist-test-slave-p39x krb5kdc[15436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:25.959579 15811 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:25.960640 15811 webserver.cc:460] Webserver started at http://127.11.158.131:33411/ using document root <none> and password file <none>
I20260926 16:37:25.961231 15811 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:25.961304 15811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:25.961501 15811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:25.963706 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "027c487a87e14b518f20e946bd4d9f0f"
format_stamp: "Formatted at 2026-09-26 16:37:25 on dist-test-slave-p39x"
server_key: "66378912f4ae1dd9cb99269205687d92"
server_key_iv: "e36c19ed196b486b8a008ed4c5da0565"
server_key_version: "encryptionkey@0"
I20260926 16:37:25.964318 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "027c487a87e14b518f20e946bd4d9f0f"
format_stamp: "Formatted at 2026-09-26 16:37:25 on dist-test-slave-p39x"
server_key: "66378912f4ae1dd9cb99269205687d92"
server_key_iv: "e36c19ed196b486b8a008ed4c5da0565"
server_key_version: "encryptionkey@0"
I20260926 16:37:25.970757 15811 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260926 16:37:25.976976 15826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:25.978905 15811 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260926 16:37:25.979081 15811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "027c487a87e14b518f20e946bd4d9f0f"
format_stamp: "Formatted at 2026-09-26 16:37:25 on dist-test-slave-p39x"
server_key: "66378912f4ae1dd9cb99269205687d92"
server_key_iv: "e36c19ed196b486b8a008ed4c5da0565"
server_key_version: "encryptionkey@0"
I20260926 16:37:25.979234 15811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:25.999895 15811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:26.003432 15811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:26.003665 15811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:26.004390 15811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:26.005304 15811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:26.005379 15811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:26.005450 15811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:26.005498 15811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:26.015089 15811 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:38535
I20260926 16:37:26.015125 15939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:38535 every 8 connection(s)
I20260926 16:37:26.016044 15811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:26.018795 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15811
I20260926 16:37:26.018950 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:26.019287 11898 external_mini_cluster.cc:1428] Setting key 4c1da338de8437f3e1b30cb82f4257b8
Sep 26 16:37:26 dist-test-slave-p39x krb5kdc[15436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:26.032909 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:26.018111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:44905 (local address 127.11.158.190:41527)
0926 16:37:26.018387 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:26.018392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:26.019358 (+ 966us) server_negotiation.cc:407] Connection header received
0926 16:37:26.021290 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:26.021294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:26.021372 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:26.021486 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:26.023225 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.023771 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.025111 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.025371 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.025473 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:26.028282 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:26.028289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:26.028295 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:26.028359 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:26.030404 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:26.030954 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:26.030961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:26.030966 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:26.031072 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:26.031407 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:26.031413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:26.031418 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:26.031713 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:26.031823 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:26.032534 (+ 711us) server_negotiation.cc:299] Negotiation successful
0926 16:37:26.032711 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":82,"threads_started":1}
I20260926 16:37:26.033947 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:26.018596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:41527 (local address 127.11.158.131:44905)
0926 16:37:26.019177 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:26.019245 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:26.021043 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:26.021684 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:26.021696 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:26.022228 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:26.023030 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:26.023057 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.023917 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.023920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:26.024966 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:26.024976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.025575 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.025578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:26.025747 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:26.026427 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:26.026444 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:26.028083 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:26.030539 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:26.030544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:26.030563 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:26.030868 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:26.031167 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:26.031169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:26.031170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:26.031313 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:26.031851 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:26.031858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:26.032204 (+ 346us) client_negotiation.cc:769] Sending connection context
0926 16:37:26.032474 (+ 270us) client_negotiation.cc:240] Negotiation successful
0926 16:37:26.032853 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":111,"threads_started":1}
I20260926 16:37:26.035277 15940 heartbeater.cc:347] Connected to a master server at 127.11.158.190:41527
I20260926 16:37:26.035516 15940 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:26.036034 15940 heartbeater.cc:511] Master 127.11.158.190:41527 requested a full tablet report, sending...
I20260926 16:37:26.037212 15481 ts_manager.cc:194] Registered new tserver with Master: 027c487a87e14b518f20e946bd4d9f0f (127.11.158.131:38535)
I20260926 16:37:26.037770 15481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:44905
I20260926 16:37:26.047124 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:37:26 dist-test-slave-p39x krb5kdc[15436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:37:26 dist-test-slave-p39x krb5kdc[15436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:26.142386 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:26.131325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.11.158.129:44609)
0926 16:37:26.132011 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:26.132017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:26.132040 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:37:26.132091 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:26.132095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:26.132302 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:26.132511 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:26.133516 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.134290 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.135212 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.135653 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.135744 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:26.138903 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:26.138913 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:26.138928 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:26.138956 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:26.140774 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:26.141169 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:26.141174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:26.141176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:26.141256 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:26.141479 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:26.141481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:26.141482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:26.141773 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:26.141948 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:26.142112 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 16:37:26.142236 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":130,"threads_started":1}
Sep 26 16:37:26 dist-test-slave-p39x krb5kdc[15436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:26.153965 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:26.145403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.11.158.190:41527)
0926 16:37:26.145558 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:26.145562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:26.145623 (+ 61us) server_negotiation.cc:407] Connection header received
0926 16:37:26.145731 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:26.145733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:26.145773 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:26.145853 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:26.146455 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.146994 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.147852 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:26.148069 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:26.148148 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:26.150754 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:26.150757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:26.150759 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:26.150808 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:26.152298 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:26.152770 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:26.152773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:26.152774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:26.152854 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:26.153084 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:26.153086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:26.153087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:26.153321 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:26.153440 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:26.153718 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 16:37:26.153840 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260926 16:37:26.154421 15481 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692
I20260926 16:37:26.155066 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15543
I20260926 16:37:26.161046 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15677
I20260926 16:37:26.167042 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15811
I20260926 16:37:26.172736 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15449
2026-09-26T16:37:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3297 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:27 dist-test-slave-p39x krb5kdc[15953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:27 dist-test-slave-p39x krb5kdc[15953](info): set up 2 sockets
Sep 26 16:37:27 dist-test-slave-p39x krb5kdc[15953](info): commencing operation
krb5kdc: starting...
W20260926 16:37:28.933387 11898 mini_kdc.cc:121] Time spent starting KDC: real 2.737s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:29 dist-test-slave-p39x krb5kdc[15953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:29.109941 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33055
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:29.235530 15966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:29.235894 15966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:29.235975 15966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:29.240381 15966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:29.240509 15966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:29.240543 15966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:29.240744 15966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:29.240818 15966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:29.247615 15966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:29.248945 15966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:29.250092 15966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:29.260715 15974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:29.260847 15966 server_base.cc:1034] running on GCE node
W20260926 16:37:29.260871 15971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:29.260746 15972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:29.261705 15966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:29.263173 15966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:29.264462 15966 hybrid_clock.cc:648] HybridClock initialized: now 1790440649264421 us; error 57 us; skew 500 ppm
Sep 26 16:37:29 dist-test-slave-p39x krb5kdc[15953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:29.272918 15966 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:29.273835 15966 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:29.274252 15966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:29.274631 15966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:29.277191 15966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "5f1eafd517894784bedfaf3580c54ead"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "e376fb61cab6841f15e0bb5fd26dd61d"
server_key_iv: "f459b017d5497a0d4bab5d018de7ebd0"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.277853 15966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "5f1eafd517894784bedfaf3580c54ead"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "e376fb61cab6841f15e0bb5fd26dd61d"
server_key_iv: "f459b017d5497a0d4bab5d018de7ebd0"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.282224 15966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:37:29.285476 15980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:29.287278 15966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:29.287513 15966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "5f1eafd517894784bedfaf3580c54ead"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "e376fb61cab6841f15e0bb5fd26dd61d"
server_key_iv: "f459b017d5497a0d4bab5d018de7ebd0"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.287710 15966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:29.325083 15966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:29.329037 15966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:29.329300 15966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:29.337920 15966 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:33055
I20260926 16:37:29.338265 16032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:33055 every 8 connection(s)
I20260926 16:37:29.338699 15966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:29.341724 16033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:29.347867 16033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead: Bootstrap starting.
I20260926 16:37:29.348734 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 15966
I20260926 16:37:29.348904 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:29.349193 11898 external_mini_cluster.cc:1428] Setting key c95cd14be09cae353fca9175f847fc37
I20260926 16:37:29.351280 16033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:29.352113 16033 log.cc:824] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:29.354560 16033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead: No bootstrap required, opened a new log
Sep 26 16:37:29 dist-test-slave-p39x krb5kdc[15953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:29.359042 16033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1eafd517894784bedfaf3580c54ead" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33055 } }
I20260926 16:37:29.359366 16033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:29.359447 16033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f1eafd517894784bedfaf3580c54ead, State: Initialized, Role: FOLLOWER
I20260926 16:37:29.360107 16033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1eafd517894784bedfaf3580c54ead" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33055 } }
I20260926 16:37:29.360626 16033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:29.360754 16033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:29.360864 16033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:29.362253 16033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1eafd517894784bedfaf3580c54ead" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33055 } }
I20260926 16:37:29.362709 16033 leader_election.cc:302] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f1eafd517894784bedfaf3580c54ead; no voters:
I20260926 16:37:29.363188 16033 leader_election.cc:288] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:29.363325 16038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:29.363763 16038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [term 1 LEADER]: Becoming Leader. State: Replica: 5f1eafd517894784bedfaf3580c54ead, State: Running, Role: LEADER
I20260926 16:37:29.364391 16038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1eafd517894784bedfaf3580c54ead" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33055 } }
I20260926 16:37:29.365116 16033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:29.366616 16039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f1eafd517894784bedfaf3580c54ead. Latest consensus state: current_term: 1 leader_uuid: "5f1eafd517894784bedfaf3580c54ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1eafd517894784bedfaf3580c54ead" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33055 } } }
I20260926 16:37:29.366766 16039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:29.366972 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:29.350695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.11.158.190:33055)
0926 16:37:29.351183 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:29.351200 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:29.351236 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:37:29.352029 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:29.352048 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:29.352555 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:29.352864 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:29.353661 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.355122 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.356073 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.356863 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.357033 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:29.359923 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:29.359930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:29.359949 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:29.359986 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:29.362343 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:29.363012 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:29.363023 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:29.363032 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:29.363191 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:29.363771 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:29.363778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:29.363783 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:29.364629 (+ 846us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:29.364850 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:29.365341 (+ 491us) server_negotiation.cc:299] Negotiation successful
0926 16:37:29.365772 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":160,"threads_started":1}
I20260926 16:37:29.367400 16040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f1eafd517894784bedfaf3580c54ead" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1eafd517894784bedfaf3580c54ead" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33055 } } }
I20260926 16:37:29.367589 16040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:29.368053 16047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:29.371097 16047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:29.377170 16047 catalog_manager.cc:1347] Generated new cluster ID: 506272edc18246079e1d36af507a2a57
I20260926 16:37:29.377262 16047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:29.395728 16047 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:29.396914 16047 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:29.408613 16047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead: Generated new TSK 0
I20260926 16:37:29.409895 16047 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:29.451785 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33055
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:29.576287 16059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:29.576853 16059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:29.576937 16059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:29.581259 16059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:29.581321 16059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:29.581387 16059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:29.586802 16059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:29.588020 16059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:29.589346 16059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:29.600694 16064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:29.600680 16067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:29.601023 16059 server_base.cc:1034] running on GCE node
W20260926 16:37:29.600680 16065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:29.601588 16059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:29.602262 16059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:29.603469 16059 hybrid_clock.cc:648] HybridClock initialized: now 1790440649603387 us; error 96 us; skew 500 ppm
Sep 26 16:37:29 dist-test-slave-p39x krb5kdc[15953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:29.607213 16059 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:29.608419 16059 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:29.608520 16059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:29.608776 16059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:29.611018 16059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "73b1b7461bca4d5ebdfd064372995207"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "0e50c4ea2d60348aa0746a0c40f67d60"
server_key_iv: "fc516a86869f327d853f98cd4e91d262"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.611661 16059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "73b1b7461bca4d5ebdfd064372995207"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "0e50c4ea2d60348aa0746a0c40f67d60"
server_key_iv: "fc516a86869f327d853f98cd4e91d262"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.616011 16059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:37:29.618893 16073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:29.620482 16059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:29.620635 16059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "73b1b7461bca4d5ebdfd064372995207"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "0e50c4ea2d60348aa0746a0c40f67d60"
server_key_iv: "fc516a86869f327d853f98cd4e91d262"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.620766 16059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:29.654680 16059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:29.658550 16059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:29.658890 16059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:29.659559 16059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:29.660524 16059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:29.660593 16059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:29.660669 16059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:29.660712 16059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:29.670990 16059 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:45215
I20260926 16:37:29.671254 16186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:45215 every 8 connection(s)
I20260926 16:37:29.671700 16059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:29.679497 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 16059
I20260926 16:37:29.679651 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:29.679973 11898 external_mini_cluster.cc:1428] Setting key 247aeec0074a1ea08a5e40266adc574a
Sep 26 16:37:29 dist-test-slave-p39x krb5kdc[15953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:29.687050 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:29.673620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:47999 (local address 127.11.158.190:33055)
0926 16:37:29.673791 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:29.673795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:29.674760 (+ 965us) server_negotiation.cc:407] Connection header received
0926 16:37:29.676187 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:29.676191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:29.676241 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:29.676321 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:29.677687 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.678306 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.679897 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.680197 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.680297 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:29.683198 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:29.683201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:29.683203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:29.683227 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:29.684881 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:29.685451 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:29.685455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:29.685456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:29.685547 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:29.685905 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:29.685910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:29.685912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:29.686116 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:29.686224 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:29.686744 (+ 520us) server_negotiation.cc:299] Negotiation successful
0926 16:37:29.686878 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 16:37:29.687945 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:29.674022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33055 (local address 127.11.158.129:47999)
0926 16:37:29.674571 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:29.674627 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:29.675956 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:29.676600 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:29.676608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:29.676997 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:29.677500 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:29.677514 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.678469 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.678472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:29.679724 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:29.679741 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.680398 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.680404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:29.680631 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:29.681329 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:29.681347 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:29.683002 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:29.685026 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:29.685032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:29.685050 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:29.685334 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:29.685642 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:29.685646 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:29.685648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:29.685803 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:29.686223 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:29.686232 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:29.686503 (+ 271us) client_negotiation.cc:769] Sending connection context
0926 16:37:29.686707 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 16:37:29.687079 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":126,"threads_started":1}
I20260926 16:37:29.689358 16187 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33055
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:37:29.689644 16187 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:29.690138 16187 heartbeater.cc:511] Master 127.11.158.190:33055 requested a full tablet report, sending...
I20260926 16:37:29.691738 15997 ts_manager.cc:194] Registered new tserver with Master: 73b1b7461bca4d5ebdfd064372995207 (127.11.158.129:45215)
I20260926 16:37:29.693104 15997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:47999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:29.716673 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33055
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:29.839679 16192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:29.839975 16192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:29.840024 16192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:29.844400 16192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:29.844463 16192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:29.844558 16192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:29.850031 16192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:29.851173 16192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:29.852238 16192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:29.862532 16198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:29.862624 16192 server_base.cc:1034] running on GCE node
W20260926 16:37:29.862519 16200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:29.862537 16197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:29.863184 16192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:29.863915 16192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:29.865198 16192 hybrid_clock.cc:648] HybridClock initialized: now 1790440649865148 us; error 54 us; skew 500 ppm
Sep 26 16:37:29 dist-test-slave-p39x krb5kdc[15953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:29.868651 16192 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:29.869426 16192 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:29.869525 16192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:29.869761 16192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:29.871892 16192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "615bd32f1f7542978aec882fad725a2e"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "cdacfcee67c98727db545fc0dfc28c26"
server_key_iv: "1211ab8933b9cb3e6d4152c14c697589"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.872567 16192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "615bd32f1f7542978aec882fad725a2e"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "cdacfcee67c98727db545fc0dfc28c26"
server_key_iv: "1211ab8933b9cb3e6d4152c14c697589"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.876479 16192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260926 16:37:29.879000 16206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:29.880271 16192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:29.880450 16192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "615bd32f1f7542978aec882fad725a2e"
format_stamp: "Formatted at 2026-09-26 16:37:29 on dist-test-slave-p39x"
server_key: "cdacfcee67c98727db545fc0dfc28c26"
server_key_iv: "1211ab8933b9cb3e6d4152c14c697589"
server_key_version: "encryptionkey@0"
I20260926 16:37:29.880573 16192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:29.892673 16192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:29.895779 16192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:29.895980 16192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:29.896638 16192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:29.897471 16192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:29.897541 16192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:29.897617 16192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:29.897668 16192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:29.908615 16192 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:44995
I20260926 16:37:29.908624 16319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:44995 every 8 connection(s)
I20260926 16:37:29.909277 16192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:29.914448 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 16192
I20260926 16:37:29.914577 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:29.914886 11898 external_mini_cluster.cc:1428] Setting key e786d6c44de3ad0df17e75eaf5e8a60c
Sep 26 16:37:29 dist-test-slave-p39x krb5kdc[15953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:37:29.925163 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:29.911082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:50931 (local address 127.11.158.190:33055)
0926 16:37:29.911223 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:29.911228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:29.912223 (+ 995us) server_negotiation.cc:407] Connection header received
0926 16:37:29.913524 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:29.913526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:29.913574 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:29.913668 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:29.915259 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.916108 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.917879 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.918205 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.918331 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:29.921370 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:29.921373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:29.921375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:29.921401 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:29.922935 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:29.923509 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:29.923513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:29.923514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:29.923612 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:29.923976 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:29.923979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:29.923981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:29.924173 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:29.924279 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:29.924852 (+ 573us) server_negotiation.cc:299] Negotiation successful
0926 16:37:29.924982 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":48}
I20260926 16:37:29.926084 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:29.911568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33055 (local address 127.11.158.130:50931)
0926 16:37:29.912054 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:29.912102 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:29.913324 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:29.913833 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:29.913840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:29.914303 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:29.915045 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:29.915068 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.916224 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.916228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:29.917736 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:29.917751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:29.918369 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:29.918372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:29.918518 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:29.919193 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:29.919209 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:29.921154 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:29.923076 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:29.923083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:29.923101 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:29.923382 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:29.923708 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:29.923712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:29.923714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:29.923869 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:29.924289 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:29.924298 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:29.924604 (+ 306us) client_negotiation.cc:769] Sending connection context
0926 16:37:29.924797 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 16:37:29.925181 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":110,"threads_started":1}
I20260926 16:37:29.927438 16320 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33055
I20260926 16:37:29.927714 16320 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:29.928231 16320 heartbeater.cc:511] Master 127.11.158.190:33055 requested a full tablet report, sending...
I20260926 16:37:29.929373 15997 ts_manager.cc:194] Registered new tserver with Master: 615bd32f1f7542978aec882fad725a2e (127.11.158.130:44995)
I20260926 16:37:29.929968 15997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:50931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:29.947788 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33055
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:30.063545 16325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:30.063810 16325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:30.063858 16325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:30.067981 16325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:30.068041 16325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:30.068107 16325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:30.073385 16325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:30.074349 16325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:30.075304 16325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:30.085079 16330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:30.085390 16333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:30.085131 16331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:30.085790 16325 server_base.cc:1034] running on GCE node
I20260926 16:37:30.086385 16325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:30.087188 16325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:30.088400 16325 hybrid_clock.cc:648] HybridClock initialized: now 1790440650088368 us; error 52 us; skew 500 ppm
Sep 26 16:37:30 dist-test-slave-p39x krb5kdc[15953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440650, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:30.091548 16325 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:30.092427 16325 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:30.092495 16325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:30.092720 16325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:30.094895 16325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "e6028a5bef9b4be28902b8ea8e748095"
format_stamp: "Formatted at 2026-09-26 16:37:30 on dist-test-slave-p39x"
server_key: "01ab7c4e46699d825c19bb02170f93e6"
server_key_iv: "a15691d17f32bb385d43ddf63eb2f82a"
server_key_version: "encryptionkey@0"
I20260926 16:37:30.095490 16325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "e6028a5bef9b4be28902b8ea8e748095"
format_stamp: "Formatted at 2026-09-26 16:37:30 on dist-test-slave-p39x"
server_key: "01ab7c4e46699d825c19bb02170f93e6"
server_key_iv: "a15691d17f32bb385d43ddf63eb2f82a"
server_key_version: "encryptionkey@0"
I20260926 16:37:30.099228 16325 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:37:30.102272 16339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:30.103888 16325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:30.104045 16325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "e6028a5bef9b4be28902b8ea8e748095"
format_stamp: "Formatted at 2026-09-26 16:37:30 on dist-test-slave-p39x"
server_key: "01ab7c4e46699d825c19bb02170f93e6"
server_key_iv: "a15691d17f32bb385d43ddf63eb2f82a"
server_key_version: "encryptionkey@0"
I20260926 16:37:30.104178 16325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:30.115985 16325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:30.119366 16325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:30.119575 16325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:30.120175 16325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:30.121097 16325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:30.121145 16325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:30.121208 16325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:30.121239 16325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:30.130505 16325 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:43287
I20260926 16:37:30.130610 16452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:43287 every 8 connection(s)
I20260926 16:37:30.131294 16325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:30.134527 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 16325
I20260926 16:37:30.134676 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:30.135041 11898 external_mini_cluster.cc:1428] Setting key 2b8156646c43b7a8763391283d25b9cc
Sep 26 16:37:30 dist-test-slave-p39x krb5kdc[15953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440650, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:30.145964 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.133250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:43737 (local address 127.11.158.190:33055)
0926 16:37:30.133422 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.133427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.134392 (+ 965us) server_negotiation.cc:407] Connection header received
0926 16:37:30.136036 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.136042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.136106 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.136180 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:30.137614 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.138142 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.139269 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.139485 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.139591 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.142184 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:30.142187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:30.142189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:30.142215 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:30.143849 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:30.144474 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:30.144477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:30.144478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:30.144558 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:30.144934 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:30.144936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:30.144937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:30.145120 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:30.145196 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.145724 (+ 528us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.145831 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:37:30.146893 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.133715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33055 (local address 127.11.158.131:43737)
0926 16:37:30.134183 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:30.134226 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:30.135784 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:30.136422 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:30.136433 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:30.136844 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:30.137475 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.137491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.138245 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.138247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.139121 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.139129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.139631 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.139634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.139774 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.140381 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:30.140402 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:30.141995 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:30.143982 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:30.143988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:30.144006 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:30.144310 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:30.144667 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:30.144669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:30.144671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:30.144839 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:30.145227 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:30.145233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:30.145488 (+ 255us) client_negotiation.cc:769] Sending connection context
0926 16:37:30.145656 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 16:37:30.145939 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260926 16:37:30.148293 16453 heartbeater.cc:347] Connected to a master server at 127.11.158.190:33055
I20260926 16:37:30.148603 16453 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:30.149093 16453 heartbeater.cc:511] Master 127.11.158.190:33055 requested a full tablet report, sending...
I20260926 16:37:30.150079 15997 ts_manager.cc:194] Registered new tserver with Master: e6028a5bef9b4be28902b8ea8e748095 (127.11.158.131:43287)
I20260926 16:37:30.150666 15997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:43737
I20260926 16:37:30.161237 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:37:30.171234 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.163296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.11.158.190:33055)
0926 16:37:30.163467 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.163472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.163644 (+ 172us) server_negotiation.cc:407] Connection header received
0926 16:37:30.163801 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.163805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.163851 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.163974 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:30.164767 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.165393 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.166311 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.166620 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.166731 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.168160 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:30.168164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:30.168166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:30.168202 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:30.169504 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:30.169933 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:30.169938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:30.169940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:30.170023 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:30.170318 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:30.170321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:30.170323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:30.170550 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:30.170696 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.170930 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.171053 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:37:30.174392 15997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:30.177511 15997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:30.194689 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.187805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.11.158.130:44995)
0926 16:37:30.188274 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.188287 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.188549 (+ 262us) server_negotiation.cc:407] Connection header received
0926 16:37:30.188696 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.188701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.188868 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.188992 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:30.190324 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.191246 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.192913 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.193701 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.193826 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.193950 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.194419 (+ 469us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.194527 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":153,"threads_started":1}
I20260926 16:37:30.194691 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.187743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:45215 (local address 127.0.0.1:49284)
0926 16:37:30.188218 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:30.188231 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:30.188311 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:30.189103 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:30.189107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:30.189121 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:30.189236 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.189244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.190613 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.190617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.192639 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.192651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.194006 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.194009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.194206 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.194252 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 16:37:30.194364 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 16:37:30.194517 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":331,"thread_start_us":93,"threads_started":1}
I20260926 16:37:30.194885 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.187037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.11.158.131:43287)
0926 16:37:30.188297 (+ 1260us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.188309 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.188360 (+ 51us) server_negotiation.cc:407] Connection header received
0926 16:37:30.188445 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.188449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.188719 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.188867 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:30.189241 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.190466 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.192801 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.193677 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.193793 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.193921 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.194615 (+ 694us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.194714 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1137,"thread_start_us":141,"threads_started":1}
I20260926 16:37:30.195235 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.188141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.11.158.129:45215)
0926 16:37:30.188460 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.188468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.188492 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:37:30.188571 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.188577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.188798 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.188954 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:30.189356 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.190466 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.192838 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.193677 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.193785 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.193921 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.194976 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.195085 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":102,"threads_started":1}
I20260926 16:37:30.195202 16465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.186941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:43287 (local address 127.0.0.1:59380)
0926 16:37:30.187780 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:30.187830 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:30.187979 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:30.188955 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:30.188960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:30.188982 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:30.189131 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.189138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.190613 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.190617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.192664 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.192670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.193991 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.193995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.194154 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.194252 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 16:37:30.194364 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 16:37:30.194517 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":309,"thread_start_us":121,"threads_started":1}
I20260926 16:37:30.195516 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.186965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44995 (local address 127.0.0.1:41726)
0926 16:37:30.188434 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:30.188448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:30.188556 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:30.189060 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:30.189063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:30.189080 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:30.190172 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.190185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.191378 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.191384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.192639 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.192651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.194013 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.194017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.194252 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.194271 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:37:30.194383 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 16:37:30.194527 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":1204,"spinlock_wait_cycles":12032,"thread_start_us":156,"threads_started":1}
I20260926 16:37:30.198271 16121 tablet_service.cc:1467] Processing CreateTablet for tablet d5899b31248e47528d3c5d83a6bdcbc8 (DEFAULT_TABLE table=test-table [id=a8b4839716254f7898be03a5dea0b016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:30.198275 16254 tablet_service.cc:1467] Processing CreateTablet for tablet d5899b31248e47528d3c5d83a6bdcbc8 (DEFAULT_TABLE table=test-table [id=a8b4839716254f7898be03a5dea0b016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:30.198275 16387 tablet_service.cc:1467] Processing CreateTablet for tablet d5899b31248e47528d3c5d83a6bdcbc8 (DEFAULT_TABLE table=test-table [id=a8b4839716254f7898be03a5dea0b016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:30.199272 16121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5899b31248e47528d3c5d83a6bdcbc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:30.199297 16387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5899b31248e47528d3c5d83a6bdcbc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:30.199292 16254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5899b31248e47528d3c5d83a6bdcbc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:30.206744 16470 tablet_bootstrap.cc:492] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: Bootstrap starting.
I20260926 16:37:30.207924 16471 tablet_bootstrap.cc:492] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: Bootstrap starting.
I20260926 16:37:30.209236 16472 tablet_bootstrap.cc:492] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Bootstrap starting.
I20260926 16:37:30.210492 16470 tablet_bootstrap.cc:654] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:30.210492 16471 tablet_bootstrap.cc:654] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:30.211074 16472 tablet_bootstrap.cc:654] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:30.211473 16470 log.cc:824] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:30.211683 16471 log.cc:824] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:30.212288 16472 log.cc:824] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:30.213521 16470 tablet_bootstrap.cc:492] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: No bootstrap required, opened a new log
I20260926 16:37:30.213749 16470 ts_tablet_manager.cc:1397] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 16:37:30.214073 16472 tablet_bootstrap.cc:492] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: No bootstrap required, opened a new log
I20260926 16:37:30.214277 16472 ts_tablet_manager.cc:1397] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:37:30.214865 16471 tablet_bootstrap.cc:492] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: No bootstrap required, opened a new log
I20260926 16:37:30.215044 16471 ts_tablet_manager.cc:1397] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 16:37:30.217639 16472 raft_consensus.cc:348] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.217905 16472 raft_consensus.cc:374] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:30.217960 16472 raft_consensus.cc:729] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615bd32f1f7542978aec882fad725a2e, State: Initialized, Role: FOLLOWER
I20260926 16:37:30.218236 16470 raft_consensus.cc:348] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.218497 16470 raft_consensus.cc:374] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:30.218434 16472 consensus_queue.cc:260] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.218560 16470 raft_consensus.cc:729] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b1b7461bca4d5ebdfd064372995207, State: Initialized, Role: FOLLOWER
I20260926 16:37:30.219051 16470 consensus_queue.cc:260] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.219172 16320 heartbeater.cc:503] Master 127.11.158.190:33055 was elected leader, sending a full tablet report...
I20260926 16:37:30.219705 16187 heartbeater.cc:503] Master 127.11.158.190:33055 was elected leader, sending a full tablet report...
I20260926 16:37:30.219985 16470 ts_tablet_manager.cc:1428] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:37:30.219566 16472 ts_tablet_manager.cc:1428] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:37:30.218891 16471 raft_consensus.cc:348] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.220487 16471 raft_consensus.cc:374] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:30.220553 16471 raft_consensus.cc:729] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6028a5bef9b4be28902b8ea8e748095, State: Initialized, Role: FOLLOWER
I20260926 16:37:30.221001 16471 consensus_queue.cc:260] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.221694 16453 heartbeater.cc:503] Master 127.11.158.190:33055 was elected leader, sending a full tablet report...
I20260926 16:37:30.221943 16471 ts_tablet_manager.cc:1428] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260926 16:37:30.382237 16454 tablet.cc:2369] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:30.410446 16321 tablet.cc:2369] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:30.423143 16188 tablet.cc:2369] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:30.543047 16478 raft_consensus.cc:482] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:30.543284 16478 raft_consensus.cc:504] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.544564 16478 leader_election.cc:288] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 615bd32f1f7542978aec882fad725a2e (127.11.158.130:44995), 73b1b7461bca4d5ebdfd064372995207 (127.11.158.129:45215)
I20260926 16:37:30.549495 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.544873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:47961 (local address 127.11.158.130:44995)
0926 16:37:30.545002 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.545006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.545314 (+ 308us) server_negotiation.cc:407] Connection header received
0926 16:37:30.545377 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.545380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.545425 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.545500 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:30.545751 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.546492 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.548218 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.548858 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.548993 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.549048 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.549287 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.549378 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:37:30.549495 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.544810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:45215 (local address 127.11.158.131:35081)
0926 16:37:30.545004 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:30.545020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:30.545120 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:30.545419 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:30.545421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:30.545433 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:30.545579 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.545585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.546620 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.546624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.548083 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.548094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.549154 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.549157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.549270 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.549287 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 16:37:30.549352 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 16:37:30.549405 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":74}
I20260926 16:37:30.549503 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.544822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44995 (local address 127.11.158.131:47961)
0926 16:37:30.545212 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:30.545226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:30.545307 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:30.545523 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:30.545526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:30.545543 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:30.545645 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.545653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.546620 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.546624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.548084 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:30.548094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.549010 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.549012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:30.549145 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.549173 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:37:30.549261 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 16:37:30.549342 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":166,"thread_start_us":102,"threads_started":1}
I20260926 16:37:30.549566 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.544970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:35081 (local address 127.11.158.129:45215)
0926 16:37:30.545098 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.545103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.545126 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:37:30.545234 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.545236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.545298 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.545387 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:30.545711 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.546492 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.548382 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.549019 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.549145 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.549178 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.549377 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.549450 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:30.550173 16141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5899b31248e47528d3c5d83a6bdcbc8" candidate_uuid: "e6028a5bef9b4be28902b8ea8e748095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b1b7461bca4d5ebdfd064372995207" is_pre_election: true
I20260926 16:37:30.550171 16274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5899b31248e47528d3c5d83a6bdcbc8" candidate_uuid: "e6028a5bef9b4be28902b8ea8e748095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615bd32f1f7542978aec882fad725a2e" is_pre_election: true
I20260926 16:37:30.550474 16274 raft_consensus.cc:2454] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6028a5bef9b4be28902b8ea8e748095 in term 0.
I20260926 16:37:30.550474 16141 raft_consensus.cc:2454] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6028a5bef9b4be28902b8ea8e748095 in term 0.
I20260926 16:37:30.550926 16342 leader_election.cc:302] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615bd32f1f7542978aec882fad725a2e, e6028a5bef9b4be28902b8ea8e748095; no voters:
I20260926 16:37:30.551224 16478 raft_consensus.cc:2781] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:30.551316 16478 raft_consensus.cc:482] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:30.551370 16478 raft_consensus.cc:3037] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:30.552407 16478 raft_consensus.cc:504] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.552734 16478 leader_election.cc:288] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [CANDIDATE]: Term 1 election: Requested vote from peers 615bd32f1f7542978aec882fad725a2e (127.11.158.130:44995), 73b1b7461bca4d5ebdfd064372995207 (127.11.158.129:45215)
I20260926 16:37:30.553057 16274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5899b31248e47528d3c5d83a6bdcbc8" candidate_uuid: "e6028a5bef9b4be28902b8ea8e748095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615bd32f1f7542978aec882fad725a2e"
I20260926 16:37:30.553151 16141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5899b31248e47528d3c5d83a6bdcbc8" candidate_uuid: "e6028a5bef9b4be28902b8ea8e748095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b1b7461bca4d5ebdfd064372995207"
I20260926 16:37:30.553169 16274 raft_consensus.cc:3037] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:30.553282 16141 raft_consensus.cc:3037] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:30.554181 16274 raft_consensus.cc:2454] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6028a5bef9b4be28902b8ea8e748095 in term 1.
I20260926 16:37:30.554334 16141 raft_consensus.cc:2454] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6028a5bef9b4be28902b8ea8e748095 in term 1.
I20260926 16:37:30.554481 16342 leader_election.cc:302] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615bd32f1f7542978aec882fad725a2e, e6028a5bef9b4be28902b8ea8e748095; no voters:
I20260926 16:37:30.554672 16478 raft_consensus.cc:2781] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:30.554914 16478 raft_consensus.cc:686] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 1 LEADER]: Becoming Leader. State: Replica: e6028a5bef9b4be28902b8ea8e748095, State: Running, Role: LEADER
I20260926 16:37:30.555233 16478 consensus_queue.cc:237] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } }
I20260926 16:37:30.558501 15997 catalog_manager.cc:5495] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6028a5bef9b4be28902b8ea8e748095 (127.11.158.131). New cstate: current_term: 1 leader_uuid: "e6028a5bef9b4be28902b8ea8e748095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6028a5bef9b4be28902b8ea8e748095" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:30.644634 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.641324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.11.158.131:43287)
0926 16:37:30.641446 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.641449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.641461 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:37:30.641531 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.641533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.641593 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.641685 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:30.641957 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.642456 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.643284 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:30.643482 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:30.643568 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:30.643823 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:30.644236 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:30.644325 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.644464 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.644511 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:37:30.650252 16274 raft_consensus.cc:1261] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 1 FOLLOWER]: Refusing update from remote peer e6028a5bef9b4be28902b8ea8e748095: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:30.650424 16141 raft_consensus.cc:1261] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 1 FOLLOWER]: Refusing update from remote peer e6028a5bef9b4be28902b8ea8e748095: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:30.650990 16478 consensus_queue.cc:1035] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615bd32f1f7542978aec882fad725a2e" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:30.651212 16480 consensus_queue.cc:1035] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b1b7461bca4d5ebdfd064372995207" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 45215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:30.658622 16485 mvcc.cc:205] Tried to move back new op lower bound from 7333644905056366592 to 7333644904676610048. Current Snapshot: MvccSnapshot[applied={T|T < 7333644905056366592}]
I20260926 16:37:30.660727 16484 mvcc.cc:205] Tried to move back new op lower bound from 7333644905056366592 to 7333644904676610048. Current Snapshot: MvccSnapshot[applied={T|T < 7333644905056366592}]
I20260926 16:37:30.668406 15996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:30.668608 15996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:30.671205 15996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f1eafd517894784bedfaf3580c54ead: Sending DeleteTablet for 3 replicas of tablet d5899b31248e47528d3c5d83a6bdcbc8
I20260926 16:37:30.672281 16254 tablet_service.cc:1514] Processing DeleteTablet for tablet d5899b31248e47528d3c5d83a6bdcbc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:30 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41726
I20260926 16:37:30.672628 16121 tablet_service.cc:1514] Processing DeleteTablet for tablet d5899b31248e47528d3c5d83a6bdcbc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:30 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49284
I20260926 16:37:30.673098 16494 tablet_replica.cc:330] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207: stopping tablet replica
I20260926 16:37:30.673230 16387 tablet_service.cc:1514] Processing DeleteTablet for tablet d5899b31248e47528d3c5d83a6bdcbc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:30 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:59380
I20260926 16:37:30.673470 16494 raft_consensus.cc:2229] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:30.673799 16494 raft_consensus.cc:2258] T d5899b31248e47528d3c5d83a6bdcbc8 P 73b1b7461bca4d5ebdfd064372995207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:30.673940 16493 tablet_replica.cc:330] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: stopping tablet replica
I20260926 16:37:30.674260 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 16059
I20260926 16:37:30.674247 16493 raft_consensus.cc:2229] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:30.674990 16493 raft_consensus.cc:2258] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:30.676430 16493 ts_tablet_manager.cc:1905] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:30.679193 16493 ts_tablet_manager.cc:1918] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:30.679296 16493 log.cc:1196] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/d5899b31248e47528d3c5d83a6bdcbc8
I20260926 16:37:30.679600 16493 ts_tablet_manager.cc:1939] T d5899b31248e47528d3c5d83a6bdcbc8 P 615bd32f1f7542978aec882fad725a2e: Deleting consensus metadata
I20260926 16:37:30.679582 16495 tablet_replica.cc:330] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: stopping tablet replica
I20260926 16:37:30.679992 16495 raft_consensus.cc:2229] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:30.680526 16495 raft_consensus.cc:2258] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:30.680939 15983 catalog_manager.cc:4841] TS 615bd32f1f7542978aec882fad725a2e (127.11.158.130:44995): tablet d5899b31248e47528d3c5d83a6bdcbc8 (table test-table [id=a8b4839716254f7898be03a5dea0b016]) successfully deleted
W20260926 16:37:30.682065 15982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:45215 (error 108)
I20260926 16:37:30.682253 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 16192
I20260926 16:37:30.685163 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.684856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:45215 (local address 127.0.0.1:49288)
0926 16:37:30.685005 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:30.685074 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:45215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 16:37:30.685333 15982 catalog_manager.cc:4584] TS 73b1b7461bca4d5ebdfd064372995207 (127.11.158.129:45215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5899b31248e47528d3c5d83a6bdcbc8: Network error: Client connection negotiation failed: client connection to 127.11.158.129:45215: connect: Connection refused (error 111)
I20260926 16:37:30.682265 16495 ts_tablet_manager.cc:1905] T d5899b31248e47528d3c5d83a6bdcbc8 P e6028a5bef9b4be28902b8ea8e748095: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:30.688555 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 16325
I20260926 16:37:30.693691 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 15966
2026-09-26T16:37:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4520 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T16:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:30Z Disabled control of system clock
I20260926 16:37:30.724794 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:42169
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:42169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 16:37:30.834192 16500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:30.834510 16500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:30.834586 16500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:30.839056 16500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:30.839136 16500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:30.839154 16500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:30.839171 16500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:30.839187 16500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:30.844188 16500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:42169
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:42169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:30.845572 16500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:30.847036 16500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:30.855813 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:30.855813 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:30.855911 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:30.856143 16500 server_base.cc:1034] running on GCE node
I20260926 16:37:30.856917 16500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:30.858093 16500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:30.859268 16500 hybrid_clock.cc:648] HybridClock initialized: now 1790440650859233 us; error 49 us; skew 500 ppm
I20260926 16:37:30.861340 16500 webserver.cc:460] Webserver started at http://127.11.158.190:35727/ using document root <none> and password file <none>
I20260926 16:37:30.861940 16500 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:30.862004 16500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:30.862231 16500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:30.864632 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "a8c16a3cb89243a5a95c55bd4e2d1431"
format_stamp: "Formatted at 2026-09-26 16:37:30 on dist-test-slave-p39x"
server_key: "b99c14397e6dc396321ef4d641c5b360"
server_key_iv: "e45fe37988779ef706567a0be5e25abc"
server_key_version: "encryptionkey@0"
I20260926 16:37:30.865203 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "a8c16a3cb89243a5a95c55bd4e2d1431"
format_stamp: "Formatted at 2026-09-26 16:37:30 on dist-test-slave-p39x"
server_key: "b99c14397e6dc396321ef4d641c5b360"
server_key_iv: "e45fe37988779ef706567a0be5e25abc"
server_key_version: "encryptionkey@0"
I20260926 16:37:30.868953 16500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:30.871536 16514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:30.872995 16500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:30.873152 16500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "a8c16a3cb89243a5a95c55bd4e2d1431"
format_stamp: "Formatted at 2026-09-26 16:37:30 on dist-test-slave-p39x"
server_key: "b99c14397e6dc396321ef4d641c5b360"
server_key_iv: "e45fe37988779ef706567a0be5e25abc"
server_key_version: "encryptionkey@0"
I20260926 16:37:30.873281 16500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:30.878100 16500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:30.878746 16500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:30.878937 16500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:30.885789 16500 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:42169
I20260926 16:37:30.885804 16566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:42169 every 8 connection(s)
I20260926 16:37:30.886744 16500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:30.890143 16567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:30.890331 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 16500
I20260926 16:37:30.890439 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:30.890748 11898 external_mini_cluster.cc:1428] Setting key 93b63e135447e9bc1834defc6bef994a
I20260926 16:37:30.896283 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:30.892139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.11.158.190:42169)
0926 16:37:30.892775 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:30.892790 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:30.892824 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:37:30.893529 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:30.893547 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:30.893894 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:30.894182 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:30.895021 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:30.895025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:30.895037 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:30.895046 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:30.895100 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:30.895116 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:30.895226 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:30.895327 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 16:37:30.895621 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":148,"threads_started":1}
I20260926 16:37:30.897104 16567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431: Bootstrap starting.
I20260926 16:37:30.899156 16567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:30.899852 16567 log.cc:824] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:30.901736 16567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431: No bootstrap required, opened a new log
I20260926 16:37:30.905639 16567 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42169 } }
I20260926 16:37:30.905890 16567 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:30.905970 16567 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c16a3cb89243a5a95c55bd4e2d1431, State: Initialized, Role: FOLLOWER
I20260926 16:37:30.906390 16567 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42169 } }
I20260926 16:37:30.906539 16567 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:30.906623 16567 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:30.906738 16567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:30.907635 16567 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42169 } }
I20260926 16:37:30.908165 16567 leader_election.cc:302] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8c16a3cb89243a5a95c55bd4e2d1431; no voters:
I20260926 16:37:30.908537 16567 leader_election.cc:288] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:30.908674 16572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:30.908927 16572 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [term 1 LEADER]: Becoming Leader. State: Replica: a8c16a3cb89243a5a95c55bd4e2d1431, State: Running, Role: LEADER
I20260926 16:37:30.909250 16572 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42169 } }
I20260926 16:37:30.909888 16567 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:30.911110 16574 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8c16a3cb89243a5a95c55bd4e2d1431. Latest consensus state: current_term: 1 leader_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42169 } } }
I20260926 16:37:30.911245 16574 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:30.911474 16573 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c16a3cb89243a5a95c55bd4e2d1431" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42169 } } }
I20260926 16:37:30.912158 16573 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:30.912106 16581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:30.915220 16581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:30.920442 16581 catalog_manager.cc:1347] Generated new cluster ID: 5cb1ee8482b74a08b386756b3939a19e
I20260926 16:37:30.920529 16581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:30.946031 16581 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:30.946380 16581 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:30.960645 16581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431: Generated new TSK 0
I20260926 16:37:30.961448 16581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:37:30.964524 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42169
--builtin_ntp_servers=127.11.158.148:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 16:37:31.084411 16591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:31.084698 16591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:31.084741 16591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:31.088809 16591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:31.088866 16591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:31.088958 16591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:31.093703 16591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:31.094808 16591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:31.096220 16591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:31.106348 16596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:31.106340 16597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:31.106436 16591 server_base.cc:1034] running on GCE node
W20260926 16:37:31.106348 16599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:31.107059 16591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:31.107743 16591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:31.108945 16591 hybrid_clock.cc:648] HybridClock initialized: now 1790440651108924 us; error 41 us; skew 500 ppm
I20260926 16:37:31.111073 16591 webserver.cc:460] Webserver started at http://127.11.158.129:43295/ using document root <none> and password file <none>
I20260926 16:37:31.111707 16591 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:31.111797 16591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:31.112040 16591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:31.114221 16591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "1ab6ba8941874b1ba0819fa39c306beb"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "07760696cca7e84b741739caa82a8432"
server_key_iv: "c8da745169487a87005996c40963e70c"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.114817 16591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "1ab6ba8941874b1ba0819fa39c306beb"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "07760696cca7e84b741739caa82a8432"
server_key_iv: "c8da745169487a87005996c40963e70c"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.118634 16591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:37:31.121269 16605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.122430 16591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:31.122567 16591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "1ab6ba8941874b1ba0819fa39c306beb"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "07760696cca7e84b741739caa82a8432"
server_key_iv: "c8da745169487a87005996c40963e70c"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.122705 16591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:31.127250 16591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:31.127851 16591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:31.128032 16591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:31.128623 16591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:31.129498 16591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:31.129565 16591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.129640 16591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:31.129683 16591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.138829 16591 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:35627
I20260926 16:37:31.138850 16718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:35627 every 8 connection(s)
I20260926 16:37:31.139744 16591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:31.141165 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 16591
I20260926 16:37:31.141302 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:31.141635 11898 external_mini_cluster.cc:1428] Setting key 2d5c2cbce68dc2615e3d13e08200ae18
I20260926 16:37:31.143767 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42169
--builtin_ntp_servers=127.11.158.148:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 16:37:31.146224 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.141701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:56689 (local address 127.11.158.190:42169)
0926 16:37:31.141845 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.141850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.143092 (+ 1242us) server_negotiation.cc:407] Connection header received
0926 16:37:31.144503 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.144506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.144549 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.144627 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.145738 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.145742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.145745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.145757 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.145778 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.145787 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.145890 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.146035 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.146092 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:37:31.147063 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.142188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:42169 (local address 127.11.158.129:56689)
0926 16:37:31.142936 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.142987 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.144254 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.144789 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.144796 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.144886 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.145471 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.145473 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.145514 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.145590 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.145875 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.145884 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.145889 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.146071 (+ 182us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.146315 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":130,"threads_started":1}
I20260926 16:37:31.148303 16719 heartbeater.cc:347] Connected to a master server at 127.11.158.190:42169
I20260926 16:37:31.148625 16719 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:31.149184 16719 heartbeater.cc:511] Master 127.11.158.190:42169 requested a full tablet report, sending...
I20260926 16:37:31.150879 16531 ts_manager.cc:194] Registered new tserver with Master: 1ab6ba8941874b1ba0819fa39c306beb (127.11.158.129:35627)
W20260926 16:37:31.260401 16722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:31.260689 16722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:31.260733 16722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:31.265007 16722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:31.265077 16722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:31.265138 16722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:31.270567 16722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:31.271727 16722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:31.273257 16722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:31.283519 16727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:31.283519 16730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:31.283520 16728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:31.283756 16722 server_base.cc:1034] running on GCE node
I20260926 16:37:31.284298 16722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:31.284986 16722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:31.286160 16722 hybrid_clock.cc:648] HybridClock initialized: now 1790440651286144 us; error 29 us; skew 500 ppm
I20260926 16:37:31.288164 16722 webserver.cc:460] Webserver started at http://127.11.158.130:40845/ using document root <none> and password file <none>
I20260926 16:37:31.288821 16722 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:31.288887 16722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:31.289108 16722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:31.291198 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "98911134017a4b059b5b0a2bb6f4a391"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "6a5390f18e868d4a6216821610cada33"
server_key_iv: "db6ab12436286f1781d17273e112d71d"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.291762 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "98911134017a4b059b5b0a2bb6f4a391"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "6a5390f18e868d4a6216821610cada33"
server_key_iv: "db6ab12436286f1781d17273e112d71d"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.295653 16722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:37:31.298267 16736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.299542 16722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:37:31.299685 16722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "98911134017a4b059b5b0a2bb6f4a391"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "6a5390f18e868d4a6216821610cada33"
server_key_iv: "db6ab12436286f1781d17273e112d71d"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.299806 16722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:31.304608 16722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:31.305272 16722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:31.305472 16722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:31.306039 16722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:31.306995 16722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:31.307067 16722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.307163 16722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:31.307214 16722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.316390 16722 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:37387
I20260926 16:37:31.316431 16849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:37387 every 8 connection(s)
I20260926 16:37:31.317351 16722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:31.320600 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 16722
I20260926 16:37:31.320730 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:31.321019 11898 external_mini_cluster.cc:1428] Setting key 4079badba4aca760483ca83c3ae0f019
I20260926 16:37:31.323251 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42169
--builtin_ntp_servers=127.11.158.148:41637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 16:37:31.324106 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.319213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:42037 (local address 127.11.158.190:42169)
0926 16:37:31.319367 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.319372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.320592 (+ 1220us) server_negotiation.cc:407] Connection header received
0926 16:37:31.322179 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.322182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.322224 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.322306 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.323654 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.323658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.323660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.323677 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.323701 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.323710 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.323799 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.323962 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.324002 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:37:31.325256 16852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.319676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:42169 (local address 127.11.158.130:42037)
0926 16:37:31.320417 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.320474 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.321966 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.322528 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.322537 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.322666 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.323294 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.323296 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.323360 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.323463 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.323802 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.323813 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.323822 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.324007 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.324274 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":104,"threads_started":1}
I20260926 16:37:31.326695 16850 heartbeater.cc:347] Connected to a master server at 127.11.158.190:42169
I20260926 16:37:31.326957 16850 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:31.327414 16850 heartbeater.cc:511] Master 127.11.158.190:42169 requested a full tablet report, sending...
I20260926 16:37:31.328567 16531 ts_manager.cc:194] Registered new tserver with Master: 98911134017a4b059b5b0a2bb6f4a391 (127.11.158.130:37387)
W20260926 16:37:31.440522 16853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:31.440794 16853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:31.440835 16853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:31.444870 16853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:31.444928 16853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:31.444985 16853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:31.449775 16853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:31.450814 16853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:31.452118 16853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:31.464412 16859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:31.464546 16853 server_base.cc:1034] running on GCE node
W20260926 16:37:31.464437 16858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:31.464421 16861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:31.465160 16853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:31.465873 16853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:31.467084 16853 hybrid_clock.cc:648] HybridClock initialized: now 1790440651467065 us; error 33 us; skew 500 ppm
I20260926 16:37:31.469209 16853 webserver.cc:460] Webserver started at http://127.11.158.131:32953/ using document root <none> and password file <none>
I20260926 16:37:31.469843 16853 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:31.469940 16853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:31.470203 16853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:31.472810 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "9cf43d1d15854124806fc891556c9a88"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "7d5e965f9a610c87ecbdc566a98800c4"
server_key_iv: "ebc1d9cbf4e6088084be810e99e39f28"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.473455 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "9cf43d1d15854124806fc891556c9a88"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "7d5e965f9a610c87ecbdc566a98800c4"
server_key_iv: "ebc1d9cbf4e6088084be810e99e39f28"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.477666 16853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 16:37:31.480511 16867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.481797 16853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:37:31.481941 16853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "9cf43d1d15854124806fc891556c9a88"
format_stamp: "Formatted at 2026-09-26 16:37:31 on dist-test-slave-p39x"
server_key: "7d5e965f9a610c87ecbdc566a98800c4"
server_key_iv: "ebc1d9cbf4e6088084be810e99e39f28"
server_key_version: "encryptionkey@0"
I20260926 16:37:31.482074 16853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:31.487219 16853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:31.487923 16853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:31.488132 16853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:31.488826 16853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:31.489809 16853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:31.489882 16853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.489951 16853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:31.489998 16853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:31.499634 16853 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:40329
I20260926 16:37:31.499691 16980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:40329 every 8 connection(s)
I20260926 16:37:31.500672 16853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:31.507525 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.502721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:37375 (local address 127.11.158.190:42169)
0926 16:37:31.502977 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.502980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.504208 (+ 1228us) server_negotiation.cc:407] Connection header received
0926 16:37:31.505657 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.505661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.505711 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.505791 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.507016 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.507025 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.507028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.507041 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.507061 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.507069 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.507174 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.507325 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.507376 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260926 16:37:31.508533 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.503219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:42169 (local address 127.11.158.131:37375)
0926 16:37:31.504040 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.504088 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.505412 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.505980 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.505987 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.506092 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.506727 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.506729 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.506775 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.506849 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.507164 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.507173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.507179 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.507377 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.507648 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":110,"threads_started":1}
I20260926 16:37:31.509994 16981 heartbeater.cc:347] Connected to a master server at 127.11.158.190:42169
I20260926 16:37:31.510243 16981 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:31.511009 16981 heartbeater.cc:511] Master 127.11.158.190:42169 requested a full tablet report, sending...
I20260926 16:37:31.511142 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 16853
I20260926 16:37:31.511253 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:31.511575 11898 external_mini_cluster.cc:1428] Setting key 5774bc75b04b26adc697ef4c83a22aee
I20260926 16:37:31.512627 16531 ts_manager.cc:194] Registered new tserver with Master: 9cf43d1d15854124806fc891556c9a88 (127.11.158.131:40329)
I20260926 16:37:31.513916 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:37:31.518072 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.516122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.11.158.190:42169)
0926 16:37:31.516266 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.516269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.516303 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:37:31.516469 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.516495 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.516533 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.516602 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.517569 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.517574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.517576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.517591 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.517613 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.517622 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.517710 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.517904 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.517959 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:37:31.521410 16531 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:31.524628 16531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:31.537678 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.534197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:37387 (local address 127.0.0.1:57390)
0926 16:37:31.534863 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.534877 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.534941 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.535941 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.535946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.535963 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.537017 (+ 1054us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.537019 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.537028 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.537055 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.537377 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.537383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.537387 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.537456 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.537487 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":104,"threads_started":1}
I20260926 16:37:31.537684 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.534134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:40329 (local address 127.0.0.1:46292)
0926 16:37:31.534562 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.534611 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.534719 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.535703 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.535710 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.535795 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.536792 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.536794 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.536814 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.536886 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.537239 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.537249 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.537255 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.537406 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.537497 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260926 16:37:31.537746 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.534262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46292 (local address 127.11.158.131:40329)
0926 16:37:31.535209 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.535226 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.535254 (+ 28us) server_negotiation.cc:407] Connection header received
0926 16:37:31.535309 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.535314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.535525 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.536254 (+ 729us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.537025 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.537032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.537045 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.537056 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.537097 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.537124 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.537269 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.537475 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.537564 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":140,"threads_started":1}
I20260926 16:37:31.538069 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.534918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:35627 (local address 127.0.0.1:36034)
0926 16:37:31.535177 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.535191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.535269 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.536047 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.536050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.536064 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.537179 (+ 1115us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.537181 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.537189 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.537207 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.537596 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.537599 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.537602 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.537673 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.537708 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260926 16:37:31.538529 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.535060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36034 (local address 127.11.158.129:35627)
0926 16:37:31.535466 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.535477 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.535496 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:37:31.535545 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.535550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.535716 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.535841 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.537340 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.537347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.537358 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.537371 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.537413 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.537448 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.537554 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.538304 (+ 750us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.538383 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":121,"threads_started":1}
I20260926 16:37:31.538854 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.534773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57390 (local address 127.11.158.130:37387)
0926 16:37:31.535515 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.535528 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.535551 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:37:31.535604 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.535609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.535793 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.536256 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.537171 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.537178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.537190 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.537202 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.537245 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.537280 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.537384 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.538549 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.538707 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":119,"threads_started":1}
I20260926 16:37:31.540863 16915 tablet_service.cc:1467] Processing CreateTablet for tablet 97f6dab6b58d41868e9f5bea807ff687 (DEFAULT_TABLE table=test-table [id=85c3d7b2e58043eabaa3620fb0e25542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:31.542034 16915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97f6dab6b58d41868e9f5bea807ff687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:31.541652 16653 tablet_service.cc:1467] Processing CreateTablet for tablet 97f6dab6b58d41868e9f5bea807ff687 (DEFAULT_TABLE table=test-table [id=85c3d7b2e58043eabaa3620fb0e25542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:31.541652 16784 tablet_service.cc:1467] Processing CreateTablet for tablet 97f6dab6b58d41868e9f5bea807ff687 (DEFAULT_TABLE table=test-table [id=85c3d7b2e58043eabaa3620fb0e25542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:31.542730 16653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97f6dab6b58d41868e9f5bea807ff687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:31.542730 16784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97f6dab6b58d41868e9f5bea807ff687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:31.548615 16998 tablet_bootstrap.cc:492] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Bootstrap starting.
I20260926 16:37:31.550571 16999 tablet_bootstrap.cc:492] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Bootstrap starting.
I20260926 16:37:31.550571 17000 tablet_bootstrap.cc:492] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: Bootstrap starting.
I20260926 16:37:31.551900 16998 tablet_bootstrap.cc:654] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:31.552775 16998 log.cc:824] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:31.552910 17000 tablet_bootstrap.cc:654] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:31.552910 16999 tablet_bootstrap.cc:654] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:31.553757 16999 log.cc:824] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:31.553757 17000 log.cc:824] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:31.554889 16998 tablet_bootstrap.cc:492] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: No bootstrap required, opened a new log
I20260926 16:37:31.555112 16998 ts_tablet_manager.cc:1397] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 16:37:31.555713 16999 tablet_bootstrap.cc:492] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: No bootstrap required, opened a new log
I20260926 16:37:31.555713 17000 tablet_bootstrap.cc:492] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: No bootstrap required, opened a new log
I20260926 16:37:31.555904 17000 ts_tablet_manager.cc:1397] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:31.555904 16999 ts_tablet_manager.cc:1397] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:31.559016 16998 raft_consensus.cc:348] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.559296 16998 raft_consensus.cc:374] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:31.559351 16998 raft_consensus.cc:729] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cf43d1d15854124806fc891556c9a88, State: Initialized, Role: FOLLOWER
I20260926 16:37:31.559808 16998 consensus_queue.cc:260] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.560530 16981 heartbeater.cc:503] Master 127.11.158.190:42169 was elected leader, sending a full tablet report...
I20260926 16:37:31.560570 17000 raft_consensus.cc:348] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.560573 16999 raft_consensus.cc:348] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.560824 16999 raft_consensus.cc:374] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:31.560824 17000 raft_consensus.cc:374] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:31.560874 16999 raft_consensus.cc:729] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98911134017a4b059b5b0a2bb6f4a391, State: Initialized, Role: FOLLOWER
I20260926 16:37:31.560874 17000 raft_consensus.cc:729] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab6ba8941874b1ba0819fa39c306beb, State: Initialized, Role: FOLLOWER
I20260926 16:37:31.560907 16998 ts_tablet_manager.cc:1428] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:37:31.561353 17000 consensus_queue.cc:260] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.561353 16999 consensus_queue.cc:260] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.562099 16719 heartbeater.cc:503] Master 127.11.158.190:42169 was elected leader, sending a full tablet report...
I20260926 16:37:31.562364 17000 ts_tablet_manager.cc:1428] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:37:31.562364 16999 ts_tablet_manager.cc:1428] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:37:31.562691 16850 heartbeater.cc:503] Master 127.11.158.190:42169 was elected leader, sending a full tablet report...
W20260926 16:37:31.568421 16851 tablet.cc:2369] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:31.641036 16720 tablet.cc:2369] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:31.751729 16982 tablet.cc:2369] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:31.845666 17006 raft_consensus.cc:482] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:31.845945 17006 raft_consensus.cc:504] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.847198 17006 leader_election.cc:288] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cf43d1d15854124806fc891556c9a88 (127.11.158.131:40329), 1ab6ba8941874b1ba0819fa39c306beb (127.11.158.129:35627)
I20260926 16:37:31.849532 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.847675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:56307 (local address 127.11.158.131:40329)
0926 16:37:31.847826 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.847829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.847845 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:31.847986 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.847992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.848041 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.848170 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.849024 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.849028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.849031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.849045 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.849071 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.849081 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.849189 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.849318 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.849378 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:37:31.849525 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.847510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:40329 (local address 127.11.158.130:56307)
0926 16:37:31.847766 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.847782 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.847878 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.848204 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.848206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.848219 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.848794 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.848796 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.848830 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.848868 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.849209 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.849214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.849218 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.849309 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.849351 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260926 16:37:31.849637 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.848076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:51677 (local address 127.11.158.129:35627)
0926 16:37:31.848200 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.848204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.848455 (+ 251us) server_negotiation.cc:407] Connection header received
0926 16:37:31.848513 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.848517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.848556 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.848642 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.849248 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.849254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.849260 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.849275 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.849298 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.849307 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.849390 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.849472 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.849508 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 16:37:31.849881 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.847920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:35627 (local address 127.11.158.130:51677)
0926 16:37:31.848311 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.848323 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:31.848440 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:31.848667 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:31.848670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:31.848681 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:31.849088 (+ 407us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:37:31.849090 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:31.849100 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:37:31.849129 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:31.849394 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:31.849396 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:31.849399 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 16:37:31.849479 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:37:31.849511 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":75,"threads_started":1}
I20260926 16:37:31.850314 16935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f6dab6b58d41868e9f5bea807ff687" candidate_uuid: "98911134017a4b059b5b0a2bb6f4a391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cf43d1d15854124806fc891556c9a88" is_pre_election: true
I20260926 16:37:31.850384 16673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f6dab6b58d41868e9f5bea807ff687" candidate_uuid: "98911134017a4b059b5b0a2bb6f4a391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ab6ba8941874b1ba0819fa39c306beb" is_pre_election: true
I20260926 16:37:31.850786 16673 raft_consensus.cc:2454] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98911134017a4b059b5b0a2bb6f4a391 in term 0.
I20260926 16:37:31.850790 16935 raft_consensus.cc:2454] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98911134017a4b059b5b0a2bb6f4a391 in term 0.
I20260926 16:37:31.851293 16739 leader_election.cc:302] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98911134017a4b059b5b0a2bb6f4a391, 9cf43d1d15854124806fc891556c9a88; no voters:
I20260926 16:37:31.851583 17006 raft_consensus.cc:2781] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:31.851675 17006 raft_consensus.cc:482] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:31.851718 17006 raft_consensus.cc:3037] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:31.852909 17006 raft_consensus.cc:504] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.853273 17006 leader_election.cc:288] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [CANDIDATE]: Term 1 election: Requested vote from peers 9cf43d1d15854124806fc891556c9a88 (127.11.158.131:40329), 1ab6ba8941874b1ba0819fa39c306beb (127.11.158.129:35627)
I20260926 16:37:31.853636 16935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f6dab6b58d41868e9f5bea807ff687" candidate_uuid: "98911134017a4b059b5b0a2bb6f4a391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cf43d1d15854124806fc891556c9a88"
I20260926 16:37:31.853749 16673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f6dab6b58d41868e9f5bea807ff687" candidate_uuid: "98911134017a4b059b5b0a2bb6f4a391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ab6ba8941874b1ba0819fa39c306beb"
I20260926 16:37:31.853749 16935 raft_consensus.cc:3037] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:31.853911 16673 raft_consensus.cc:3037] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:31.854882 16935 raft_consensus.cc:2454] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98911134017a4b059b5b0a2bb6f4a391 in term 1.
I20260926 16:37:31.855052 16673 raft_consensus.cc:2454] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98911134017a4b059b5b0a2bb6f4a391 in term 1.
I20260926 16:37:31.855204 16739 leader_election.cc:302] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98911134017a4b059b5b0a2bb6f4a391, 9cf43d1d15854124806fc891556c9a88; no voters:
I20260926 16:37:31.855400 17006 raft_consensus.cc:2781] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:31.855640 17006 raft_consensus.cc:686] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 1 LEADER]: Becoming Leader. State: Replica: 98911134017a4b059b5b0a2bb6f4a391, State: Running, Role: LEADER
I20260926 16:37:31.856031 17006 consensus_queue.cc:237] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } }
I20260926 16:37:31.859481 16529 catalog_manager.cc:5495] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98911134017a4b059b5b0a2bb6f4a391 (127.11.158.130). New cstate: current_term: 1 leader_uuid: "98911134017a4b059b5b0a2bb6f4a391" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98911134017a4b059b5b0a2bb6f4a391" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:31.905041 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.903311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.11.158.130:37387)
0926 16:37:31.903464 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:31.903469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:31.903483 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:37:31.903672 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:31.903675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:31.903742 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:31.903837 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:31.904615 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:31.904619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:31.904622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:31.904635 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:31.904657 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:31.904674 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:31.904776 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:31.904850 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 16:37:31.904894 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:37:31.911492 16673 raft_consensus.cc:1261] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 1 FOLLOWER]: Refusing update from remote peer 98911134017a4b059b5b0a2bb6f4a391: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:31.911492 16935 raft_consensus.cc:1261] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 1 FOLLOWER]: Refusing update from remote peer 98911134017a4b059b5b0a2bb6f4a391: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:31.912297 17009 consensus_queue.cc:1035] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cf43d1d15854124806fc891556c9a88" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:31.912528 17006 consensus_queue.cc:1035] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab6ba8941874b1ba0819fa39c306beb" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:31.927884 16529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:31.928040 16529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:31.930528 16529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8c16a3cb89243a5a95c55bd4e2d1431: Sending DeleteTablet for 3 replicas of tablet 97f6dab6b58d41868e9f5bea807ff687
I20260926 16:37:31.931340 16784 tablet_service.cc:1514] Processing DeleteTablet for tablet 97f6dab6b58d41868e9f5bea807ff687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:31 UTC) from {username='slave'} at 127.0.0.1:57390
I20260926 16:37:31.931442 16915 tablet_service.cc:1514] Processing DeleteTablet for tablet 97f6dab6b58d41868e9f5bea807ff687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:31 UTC) from {username='slave'} at 127.0.0.1:46292
I20260926 16:37:31.931555 16653 tablet_service.cc:1514] Processing DeleteTablet for tablet 97f6dab6b58d41868e9f5bea807ff687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:31 UTC) from {username='slave'} at 127.0.0.1:36034
I20260926 16:37:31.931979 17023 tablet_replica.cc:330] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb: stopping tablet replica
I20260926 16:37:31.931989 17022 tablet_replica.cc:330] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: stopping tablet replica
I20260926 16:37:31.932271 17024 tablet_replica.cc:330] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: stopping tablet replica
I20260926 16:37:31.932304 17022 raft_consensus.cc:2229] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:31.932559 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 16591
I20260926 16:37:31.932839 17023 raft_consensus.cc:2229] T 97f6dab6b58d41868e9f5bea807ff687 P 1ab6ba8941874b1ba0819fa39c306beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:31.933019 17024 raft_consensus.cc:2229] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:31.933341 17022 raft_consensus.cc:2258] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:31.933384 17024 raft_consensus.cc:2258] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:31.935022 17024 ts_tablet_manager.cc:1905] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:31.935256 17022 ts_tablet_manager.cc:1905] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:31.938159 17024 ts_tablet_manager.cc:1918] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:31.938349 17024 log.cc:1196] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/97f6dab6b58d41868e9f5bea807ff687
I20260926 16:37:31.938766 17024 ts_tablet_manager.cc:1939] T 97f6dab6b58d41868e9f5bea807ff687 P 9cf43d1d15854124806fc891556c9a88: Deleting consensus metadata
I20260926 16:37:31.938791 17022 ts_tablet_manager.cc:1918] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:31.938887 17022 log.cc:1196] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/97f6dab6b58d41868e9f5bea807ff687
I20260926 16:37:31.939227 17022 ts_tablet_manager.cc:1939] T 97f6dab6b58d41868e9f5bea807ff687 P 98911134017a4b059b5b0a2bb6f4a391: Deleting consensus metadata
I20260926 16:37:31.940387 16517 catalog_manager.cc:4841] TS 9cf43d1d15854124806fc891556c9a88 (127.11.158.131:40329): tablet 97f6dab6b58d41868e9f5bea807ff687 (table test-table [id=85c3d7b2e58043eabaa3620fb0e25542]) successfully deleted
I20260926 16:37:31.940514 16518 catalog_manager.cc:4841] TS 98911134017a4b059b5b0a2bb6f4a391 (127.11.158.130:37387): tablet 97f6dab6b58d41868e9f5bea807ff687 (table test-table [id=85c3d7b2e58043eabaa3620fb0e25542]) successfully deleted
W20260926 16:37:31.941217 16517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:35627 (error 108)
I20260926 16:37:31.941927 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 16722
I20260926 16:37:31.942297 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:31.941938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:35627 (local address 127.0.0.1:36046)
0926 16:37:31.942109 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:31.942191 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 16:37:31.944304 16517 catalog_manager.cc:4584] TS 1ab6ba8941874b1ba0819fa39c306beb (127.11.158.129:35627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97f6dab6b58d41868e9f5bea807ff687: Network error: Client connection negotiation failed: client connection to 127.11.158.129:35627: connect: Connection refused (error 111)
I20260926 16:37:31.947947 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 16853
I20260926 16:37:31.953715 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 16500
2026-09-26T16:37:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1258 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 16:37:31.975361 11898 mini_oidc.cc:152] Starting JWKS server
I20260926 16:37:31.977145 11898 webserver.cc:460] Webserver started at https://127.0.0.1:35609/ using document root <none> and password file <none>
I20260926 16:37:31.977331 11898 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 16:37:31.977650 11898 webserver.cc:460] Webserver started at http://127.0.0.1:38859/ using document root <none> and password file <none>
2026-09-26T16:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:31Z Disabled control of system clock
I20260926 16:37:31.987473 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:42263
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:43975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:42263
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35609/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/testchainca.cert with env {}
W20260926 16:37:32.114557 17031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:32.114897 17031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:32.114988 17031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:32.119349 17031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:32.119462 17031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:32.119494 17031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:32.119563 17031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:32.119601 17031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:37:32.119654 17031 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:37:32.119689 17031 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35609/jwks.json
I20260926 16:37:32.125324 17031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:42263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:42263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35609/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:32.126438 17031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:32.127460 17031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:32.136412 17036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:32.136477 17031 server_base.cc:1034] running on GCE node
W20260926 16:37:32.136502 17039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:32.136415 17037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:32.137398 17031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:32.138659 17031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:32.139832 17031 hybrid_clock.cc:648] HybridClock initialized: now 1790440652139790 us; error 57 us; skew 500 ppm
I20260926 16:37:32.141907 17031 webserver.cc:460] Webserver started at http://127.11.158.190:45017/ using document root <none> and password file <none>
I20260926 16:37:32.142498 17031 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:32.142567 17031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:32.142793 17031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:32.144989 17031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "f3e3bf0f4d6443f09b99255b305ad617"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "6502fe7b8dc87345d86697027ad5d626"
server_key_iv: "630b265aadf8aeb155bc2c8e44f318c3"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.145591 17031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "f3e3bf0f4d6443f09b99255b305ad617"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "6502fe7b8dc87345d86697027ad5d626"
server_key_iv: "630b265aadf8aeb155bc2c8e44f318c3"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.149446 17031 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:37:32.151953 17045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:32.153309 17031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:32.153442 17031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "f3e3bf0f4d6443f09b99255b305ad617"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "6502fe7b8dc87345d86697027ad5d626"
server_key_iv: "630b265aadf8aeb155bc2c8e44f318c3"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.153584 17031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:32.170550 17031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:32.171293 17031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:32.171494 17031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:32.178913 17031 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:42263
I20260926 16:37:32.178918 17097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:42263 every 8 connection(s)
I20260926 16:37:32.179908 17031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:32.183146 17098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:32.185484 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17031
I20260926 16:37:32.185607 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:32.185914 11898 external_mini_cluster.cc:1428] Setting key 4f28d451a7e2596ff24cbd2850fffc0c
I20260926 16:37:32.189638 17098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617: Bootstrap starting.
I20260926 16:37:32.192160 17098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:32.193089 17098 log.cc:824] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:32.195634 17098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617: No bootstrap required, opened a new log
I20260926 16:37:32.195960 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:32.187435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.11.158.190:42263)
0926 16:37:32.188096 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:32.188111 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:32.188147 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:37:32.188846 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:32.188866 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:32.189273 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:32.189632 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:32.190083 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.191059 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.191977 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.192575 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.192753 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.193719 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:32.193725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:32.193738 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:32.194267 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:32.194342 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:32.194370 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:32.194480 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:32.194660 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 16:37:32.194948 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":132,"threads_started":1}
I20260926 16:37:32.198971 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:37:32.200208 17098 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e3bf0f4d6443f09b99255b305ad617" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42263 } }
I20260926 16:37:32.200549 17098 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:32.200650 17098 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e3bf0f4d6443f09b99255b305ad617, State: Initialized, Role: FOLLOWER
I20260926 16:37:32.201113 17098 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e3bf0f4d6443f09b99255b305ad617" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42263 } }
I20260926 16:37:32.201282 17098 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:32.201377 17098 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:32.201506 17098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:32.202798 17098 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e3bf0f4d6443f09b99255b305ad617" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42263 } }
I20260926 16:37:32.203182 17098 leader_election.cc:302] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3e3bf0f4d6443f09b99255b305ad617; no voters:
I20260926 16:37:32.203547 17098 leader_election.cc:288] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:32.203663 17103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:32.203920 17103 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [term 1 LEADER]: Becoming Leader. State: Replica: f3e3bf0f4d6443f09b99255b305ad617, State: Running, Role: LEADER
I20260926 16:37:32.204241 17103 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e3bf0f4d6443f09b99255b305ad617" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42263 } }
I20260926 16:37:32.204901 17098 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:32.206195 17105 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e3bf0f4d6443f09b99255b305ad617. Latest consensus state: current_term: 1 leader_uuid: "f3e3bf0f4d6443f09b99255b305ad617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e3bf0f4d6443f09b99255b305ad617" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42263 } } }
I20260926 16:37:32.206333 17105 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:32.206846 17112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:32.207108 17104 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3e3bf0f4d6443f09b99255b305ad617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e3bf0f4d6443f09b99255b305ad617" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42263 } } }
I20260926 16:37:32.207242 17104 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:32.210245 17112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:32.216306 17112 catalog_manager.cc:1347] Generated new cluster ID: 4afe1a83683b4513a18f8ad93ad113cd
I20260926 16:37:32.216418 17112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:32.238445 17112 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:32.239837 17112 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:32.251171 17112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3e3bf0f4d6443f09b99255b305ad617: Generated new TSK 0
I20260926 16:37:32.252116 17112 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 16:37:32.279148 17128 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 16:37:32.279277 17128 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 16:37:32.279800 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:32.262380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.11.158.190:42263)
0926 16:37:32.262531 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:32.262535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:32.262793 (+ 258us) server_negotiation.cc:407] Connection header received
0926 16:37:32.262976 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:32.262978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:32.263029 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:32.263120 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:37:32.263417 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.263970 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.265257 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.265491 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.265572 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.265965 (+ 393us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:37:32.279428 (+ 13463us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 16:37:32.279688 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54374: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 16:37:32.279906 17101 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54374: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 16:37:32.281427 17127 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:32.262296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:42263 (local address 127.0.0.1:54374)
0926 16:37:32.262656 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:32.262672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:32.262812 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:32.263169 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:32.263172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:32.263181 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:37:32.263308 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:32.263316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.264096 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.264099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:32.265119 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:32.265131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.265616 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.265620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:32.265717 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.265776 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:37:32.279704 (+ 13928us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 16:37:32.279984 (+ 280us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.158.190:42263: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":109,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.158.190:42263: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 16:37:32.282385 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17031
2026-09-26T16:37:32Z chronyd exiting
I20260926 16:37:32.320977 11898 test_util.cc:175] -----------------------------------------------
I20260926 16:37:32.321079 11898 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440622937842-11898-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (347 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 16:37:32.322167 11898 mini_oidc.cc:152] Starting JWKS server
I20260926 16:37:32.323089 11898 webserver.cc:460] Webserver started at https://127.0.0.1:43031/ using document root <none> and password file <none>
I20260926 16:37:32.323179 11898 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 16:37:32.323541 11898 webserver.cc:460] Webserver started at http://127.0.0.1:36417/ using document root <none> and password file <none>
2026-09-26T16:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:32Z Disabled control of system clock
I20260926 16:37:32.333211 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34591
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:36301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:34591
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43031/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/testchainca.cert with env {}
W20260926 16:37:32.446978 17134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:32.447280 17134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:32.447322 17134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:32.451324 17134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:32.451385 17134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:32.451400 17134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:32.451416 17134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:32.451432 17134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:37:32.451448 17134 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:37:32.451462 17134 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43031/jwks.json
I20260926 16:37:32.456845 17134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:36301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:34591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43031/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:32.457881 17134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:32.458838 17134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:32.467051 17139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:32.467089 17140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:32.467234 17134 server_base.cc:1034] running on GCE node
W20260926 16:37:32.467108 17142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:32.467900 17134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:32.469034 17134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:32.470219 17134 hybrid_clock.cc:648] HybridClock initialized: now 1790440652470189 us; error 43 us; skew 500 ppm
I20260926 16:37:32.471998 17134 webserver.cc:460] Webserver started at http://127.11.158.190:45681/ using document root <none> and password file <none>
I20260926 16:37:32.472649 17134 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:32.472741 17134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:32.472965 17134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:32.474923 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "a57b958737c04a888ef4d66bb0f551ea"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "48d24c6aaac246b052a4e536d22dcba6"
server_key_iv: "965ca61d93c90c6e4f841295769397ff"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.475488 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "a57b958737c04a888ef4d66bb0f551ea"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "48d24c6aaac246b052a4e536d22dcba6"
server_key_iv: "965ca61d93c90c6e4f841295769397ff"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.478965 17134 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260926 16:37:32.481308 17148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:32.482527 17134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:32.482664 17134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "a57b958737c04a888ef4d66bb0f551ea"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "48d24c6aaac246b052a4e536d22dcba6"
server_key_iv: "965ca61d93c90c6e4f841295769397ff"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.482779 17134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:32.496073 17134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:32.496778 17134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:32.496975 17134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:32.504464 17134 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:34591
I20260926 16:37:32.504498 17200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:34591 every 8 connection(s)
I20260926 16:37:32.505501 17134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:32.508703 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:32.509316 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17134
I20260926 16:37:32.509438 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:32.509755 11898 external_mini_cluster.cc:1428] Setting key 62f8664080e86c9a788ecf1cf807e18c
I20260926 16:37:32.514773 17201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea: Bootstrap starting.
I20260926 16:37:32.516979 17201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:32.517688 17201 log.cc:824] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:32.518924 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:32.511076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.11.158.190:34591)
0926 16:37:32.511708 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:32.511723 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:32.511756 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:37:32.512480 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:32.512499 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:32.512884 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:32.513205 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:32.513604 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.514524 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.515481 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.515933 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.516094 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.516970 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:32.516975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:32.516987 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:32.517418 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:32.517470 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:32.517490 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:32.517596 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:32.517726 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 16:37:32.518108 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":127,"threads_started":1}
I20260926 16:37:32.519660 17201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea: No bootstrap required, opened a new log
I20260926 16:37:32.521576 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:37:32.523636 17201 raft_consensus.cc:348] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a57b958737c04a888ef4d66bb0f551ea" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34591 } }
I20260926 16:37:32.523885 17201 raft_consensus.cc:374] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:32.523963 17201 raft_consensus.cc:729] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a57b958737c04a888ef4d66bb0f551ea, State: Initialized, Role: FOLLOWER
I20260926 16:37:32.524493 17201 consensus_queue.cc:260] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a57b958737c04a888ef4d66bb0f551ea" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34591 } }
I20260926 16:37:32.524657 17201 raft_consensus.cc:388] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:32.524742 17201 raft_consensus.cc:482] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:32.524840 17201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:32.525833 17201 raft_consensus.cc:504] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a57b958737c04a888ef4d66bb0f551ea" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34591 } }
I20260926 16:37:32.526173 17201 leader_election.cc:302] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a57b958737c04a888ef4d66bb0f551ea; no voters:
I20260926 16:37:32.526502 17201 leader_election.cc:288] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:32.526628 17206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:32.526865 17206 raft_consensus.cc:686] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [term 1 LEADER]: Becoming Leader. State: Replica: a57b958737c04a888ef4d66bb0f551ea, State: Running, Role: LEADER
I20260926 16:37:32.527168 17206 consensus_queue.cc:237] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a57b958737c04a888ef4d66bb0f551ea" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34591 } }
I20260926 16:37:32.527731 17201 sys_catalog.cc:564] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:32.528887 17208 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader a57b958737c04a888ef4d66bb0f551ea. Latest consensus state: current_term: 1 leader_uuid: "a57b958737c04a888ef4d66bb0f551ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a57b958737c04a888ef4d66bb0f551ea" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34591 } } }
I20260926 16:37:32.528908 17207 sys_catalog.cc:455] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a57b958737c04a888ef4d66bb0f551ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a57b958737c04a888ef4d66bb0f551ea" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34591 } } }
I20260926 16:37:32.529047 17208 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:32.529047 17207 sys_catalog.cc:458] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:32.529382 17215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:32.532547 17215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:32.538074 17215 catalog_manager.cc:1347] Generated new cluster ID: a1811144207447c9aa0da6e5111d8f4c
I20260926 16:37:32.538142 17215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:32.576475 17215 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:32.577637 17215 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:32.591856 17215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a57b958737c04a888ef4d66bb0f551ea: Generated new TSK 0
I20260926 16:37:32.592577 17215 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 16:37:32.624773 17231 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 16:37:32.624868 17231 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 16:37:32.625419 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:32.609551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56588 (local address 127.11.158.190:34591)
0926 16:37:32.609696 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:32.609701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:32.609840 (+ 139us) server_negotiation.cc:407] Connection header received
0926 16:37:32.609975 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:32.609979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:32.610038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:32.610167 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:37:32.610366 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.611177 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.612061 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.612433 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.612538 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.612814 (+ 276us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:37:32.625010 (+ 12196us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 16:37:32.625277 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 16:37:32.625483 17204 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 16:37:32.625545 17230 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:32.609410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34591 (local address 127.0.0.1:56588)
0926 16:37:32.609758 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:32.609770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:32.609866 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:32.610167 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:32.610169 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:32.610174 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:37:32.610270 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:32.610274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.611314 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.611317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:32.611947 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:32.611954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.612556 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.612559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:32.612621 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.612643 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:37:32.625221 (+ 12578us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 16:37:32.625362 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.158.190:34591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43031/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":130,"threads_started":1}
I20260926 16:37:32.626250 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17134
2026-09-26T16:37:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (343 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 16:37:32.665782 11898 mini_oidc.cc:152] Starting JWKS server
I20260926 16:37:32.666658 11898 webserver.cc:460] Webserver started at https://127.0.0.1:43667/ using document root <none> and password file <none>
I20260926 16:37:32.666783 11898 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 16:37:32.667085 11898 webserver.cc:460] Webserver started at http://127.0.0.1:45367/ using document root <none> and password file <none>
2026-09-26T16:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:32Z Disabled control of system clock
I20260926 16:37:32.676024 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33819
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:35475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33819
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43667/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 16:37:32.787806 17237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:32.788074 17237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:32.788120 17237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:32.792249 17237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:32.792311 17237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:32.792326 17237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:32.792397 17237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:32.792429 17237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:37:32.792464 17237 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:37:32.792511 17237 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43667/jwks.json
I20260926 16:37:32.797259 17237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43667/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:32.798336 17237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:32.799712 17237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:32.808635 17243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:32.808605 17245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:32.808606 17242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:32.808674 17237 server_base.cc:1034] running on GCE node
I20260926 16:37:32.809451 17237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:32.810599 17237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:32.811796 17237 hybrid_clock.cc:648] HybridClock initialized: now 1790440652811770 us; error 39 us; skew 500 ppm
I20260926 16:37:32.813786 17237 webserver.cc:460] Webserver started at http://127.11.158.190:45801/ using document root <none> and password file <none>
I20260926 16:37:32.814378 17237 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:32.814471 17237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:32.814710 17237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:32.816870 17237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "f8cb6ad5f6e4466f85bb2619d714869a"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "e2929bd445952f01f3aa8b54db3957f9"
server_key_iv: "73e1a51a83dad0a1080616511c1f06d9"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.817445 17237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "f8cb6ad5f6e4466f85bb2619d714869a"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "e2929bd445952f01f3aa8b54db3957f9"
server_key_iv: "73e1a51a83dad0a1080616511c1f06d9"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.821146 17237 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:32.823673 17251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:32.825107 17237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:32.825256 17237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "f8cb6ad5f6e4466f85bb2619d714869a"
format_stamp: "Formatted at 2026-09-26 16:37:32 on dist-test-slave-p39x"
server_key: "e2929bd445952f01f3aa8b54db3957f9"
server_key_iv: "73e1a51a83dad0a1080616511c1f06d9"
server_key_version: "encryptionkey@0"
I20260926 16:37:32.825378 17237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:32.843941 17237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:32.844882 17237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:32.845105 17237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:32.851967 17237 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:33819
I20260926 16:37:32.851979 17303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:33819 every 8 connection(s)
I20260926 16:37:32.853042 17237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:32.855857 17304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:32.861447 17304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a: Bootstrap starting.
I20260926 16:37:32.862262 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17237
I20260926 16:37:32.862367 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:32.862692 11898 external_mini_cluster.cc:1428] Setting key c8b8b1fe6fbf052bd980a17ef1137dd3
I20260926 16:37:32.864270 17304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:32.865043 17304 log.cc:824] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:32.867043 17304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a: No bootstrap required, opened a new log
I20260926 16:37:32.870891 17304 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33819 } }
I20260926 16:37:32.871205 17304 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:32.871296 17304 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cb6ad5f6e4466f85bb2619d714869a, State: Initialized, Role: FOLLOWER
I20260926 16:37:32.871742 17304 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33819 } }
I20260926 16:37:32.871955 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:32.863777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37782 (local address 127.11.158.190:33819)
0926 16:37:32.864471 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:32.864487 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:32.864521 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:37:32.865145 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:32.865187 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:32.865597 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:32.865926 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:32.866347 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.867418 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.868359 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.868860 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.869054 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.869993 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:32.869999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:32.870013 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:32.870420 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:32.870482 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:32.870503 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:32.870616 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:32.870773 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 16:37:32.871039 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
I20260926 16:37:32.872022 17304 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:32.872107 17304 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:32.872232 17304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:32.873486 17304 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33819 } }
I20260926 16:37:32.873857 17304 leader_election.cc:302] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8cb6ad5f6e4466f85bb2619d714869a; no voters:
I20260926 16:37:32.874202 17304 leader_election.cc:288] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:32.874564 17309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:32.874814 17309 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [term 1 LEADER]: Becoming Leader. State: Replica: f8cb6ad5f6e4466f85bb2619d714869a, State: Running, Role: LEADER
I20260926 16:37:32.875289 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:37:32.875689 17304 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:32.875150 17309 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33819 } }
I20260926 16:37:32.877794 17310 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33819 } } }
I20260926 16:37:32.877952 17310 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:32.878046 17311 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8cb6ad5f6e4466f85bb2619d714869a. Latest consensus state: current_term: 1 leader_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8cb6ad5f6e4466f85bb2619d714869a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33819 } } }
I20260926 16:37:32.878157 17311 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a [sys.catalog]: This master's current role is: LEADER
W20260926 16:37:32.880326 17323 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:37:32.880470 17323 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:37:32.880550 17324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:32.881274 17324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:32.886420 17324 catalog_manager.cc:1347] Generated new cluster ID: 0ec26027c45b4c47b1daf9942ec7dfc6
I20260926 16:37:32.886488 17324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:32.895995 17324 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:32.897282 17324 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:32.901521 17324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8cb6ad5f6e4466f85bb2619d714869a: Generated new TSK 0
I20260926 16:37:32.902393 17324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:37:32.918174 17307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:32.912973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37798 (local address 127.11.158.190:33819)
0926 16:37:32.913112 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:32.913119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:32.913298 (+ 179us) server_negotiation.cc:407] Connection header received
0926 16:37:32.913465 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:32.913467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:32.913514 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:32.913604 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:37:32.913913 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.914455 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.915725 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.915942 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.916022 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.916397 (+ 375us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:37:32.917792 (+ 1395us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 16:37:32.918013 (+ 221us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37798: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260926 16:37:32.918247 17307 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37798: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 16:37:32.919981 17333 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:32.912891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33819 (local address 127.0.0.1:37798)
0926 16:37:32.913202 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:32.913218 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:32.913338 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:32.913648 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:32.913651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:32.913659 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:37:32.913779 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:32.913785 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.914599 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.914603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:32.915597 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:32.915608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:32.916063 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:32.916067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:32.916157 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:32.916186 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:37:32.917960 (+ 1774us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 16:37:32.919806 (+ 1846us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.158.190:33819: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43667/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":1645,"thread_start_us":108,"threads_started":1}
I20260926 16:37:32.920573 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17237
E20260926 16:37:32.931051 17334 webserver.cc:533] Webserver: sslize failed:
E20260926 16:37:32.931166 17334 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T16:37:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (294 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T16:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:32Z Disabled control of system clock
I20260926 16:37:32.968904 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:43755
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:43755
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/testchainca.cert with env {}
W20260926 16:37:33.078811 17338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:33.079097 17338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:33.079137 17338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:33.083302 17338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:33.083364 17338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:33.083380 17338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:33.083398 17338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:33.083413 17338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:37:33.083429 17338 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:37:33.083443 17338 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 16:37:33.088879 17338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:43755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:43755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:33.089895 17338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:33.090852 17338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:33.099004 17344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:33.099001 17346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:33.099013 17343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:33.099378 17338 server_base.cc:1034] running on GCE node
I20260926 16:37:33.099963 17338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:33.101145 17338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:33.102353 17338 hybrid_clock.cc:648] HybridClock initialized: now 1790440653102302 us; error 63 us; skew 500 ppm
I20260926 16:37:33.104584 17338 webserver.cc:460] Webserver started at http://127.11.158.190:38749/ using document root <none> and password file <none>
I20260926 16:37:33.105180 17338 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:33.105268 17338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:33.105497 17338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:33.107586 17338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "c4b460ce20f8468a95265abe547d28b6"
format_stamp: "Formatted at 2026-09-26 16:37:33 on dist-test-slave-p39x"
server_key: "3303ea070597e66aaf69b5186559be5a"
server_key_iv: "42cc8fd0b91b59de50cf40366c0e7a63"
server_key_version: "encryptionkey@0"
I20260926 16:37:33.108165 17338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "c4b460ce20f8468a95265abe547d28b6"
format_stamp: "Formatted at 2026-09-26 16:37:33 on dist-test-slave-p39x"
server_key: "3303ea070597e66aaf69b5186559be5a"
server_key_iv: "42cc8fd0b91b59de50cf40366c0e7a63"
server_key_version: "encryptionkey@0"
I20260926 16:37:33.111845 17338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:33.114452 17352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:33.115731 17338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:37:33.115871 17338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "c4b460ce20f8468a95265abe547d28b6"
format_stamp: "Formatted at 2026-09-26 16:37:33 on dist-test-slave-p39x"
server_key: "3303ea070597e66aaf69b5186559be5a"
server_key_iv: "42cc8fd0b91b59de50cf40366c0e7a63"
server_key_version: "encryptionkey@0"
I20260926 16:37:33.115988 17338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:33.134493 17338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:33.135229 17338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:33.135437 17338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:33.142627 17338 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:43755
I20260926 16:37:33.142649 17404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:43755 every 8 connection(s)
I20260926 16:37:33.143555 17338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:33.144543 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17338
I20260926 16:37:33.144657 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:33.144963 11898 external_mini_cluster.cc:1428] Setting key 1929c02d2fbdcc4085439f324f739470
I20260926 16:37:33.147008 17405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:33.152598 17405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6: Bootstrap starting.
I20260926 16:37:33.154014 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:33.146322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.11.158.190:43755)
0926 16:37:33.146958 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:33.146973 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:33.147007 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:37:33.147703 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:33.147719 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:33.148106 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:33.148480 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:33.148894 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:33.149838 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:33.150738 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:33.151204 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:33.151367 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:33.152264 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:33.152269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:33.152280 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:37:33.152762 (+ 482us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:33.152818 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:37:33.152840 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:33.152945 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:33.153085 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 16:37:33.153341 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":147,"threads_started":1}
I20260926 16:37:33.154856 17405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:33.155720 17405 log.cc:824] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:33.156777 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:37:33.157833 17405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6: No bootstrap required, opened a new log
I20260926 16:37:33.161706 17405 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b460ce20f8468a95265abe547d28b6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43755 } }
I20260926 16:37:33.161938 17405 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:33.162017 17405 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b460ce20f8468a95265abe547d28b6, State: Initialized, Role: FOLLOWER
I20260926 16:37:33.162462 17405 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b460ce20f8468a95265abe547d28b6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43755 } }
I20260926 16:37:33.162616 17405 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:33.162699 17405 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:33.162794 17405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:33.163650 17405 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b460ce20f8468a95265abe547d28b6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43755 } }
I20260926 16:37:33.163978 17405 leader_election.cc:302] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4b460ce20f8468a95265abe547d28b6; no voters:
I20260926 16:37:33.164295 17405 leader_election.cc:288] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:33.164443 17410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:33.164656 17410 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [term 1 LEADER]: Becoming Leader. State: Replica: c4b460ce20f8468a95265abe547d28b6, State: Running, Role: LEADER
I20260926 16:37:33.164948 17410 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b460ce20f8468a95265abe547d28b6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43755 } }
I20260926 16:37:33.165464 17405 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:33.166008 17410 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4b460ce20f8468a95265abe547d28b6. Latest consensus state: current_term: 1 leader_uuid: "c4b460ce20f8468a95265abe547d28b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b460ce20f8468a95265abe547d28b6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43755 } } }
I20260926 16:37:33.166136 17410 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:33.166492 17412 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4b460ce20f8468a95265abe547d28b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b460ce20f8468a95265abe547d28b6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43755 } } }
I20260926 16:37:33.166592 17412 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:33.166862 17416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:33.169929 17416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:33.175696 17416 catalog_manager.cc:1347] Generated new cluster ID: ef69e28308124ddcb3183fef58548dbf
I20260926 16:37:33.175767 17416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:33.189143 17416 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:33.190001 17416 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:33.199604 17416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4b460ce20f8468a95265abe547d28b6: Generated new TSK 0
I20260926 16:37:33.200317 17416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:37:33.680202 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:33.208530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.11.158.190:43755)
0926 16:37:33.208673 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:33.208678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:33.208830 (+ 152us) server_negotiation.cc:407] Connection header received
0926 16:37:33.209004 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:33.209007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:33.209062 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:33.209179 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:37:33.209371 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:33.210181 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:33.211024 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:33.211361 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:33.211460 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:33.211738 (+ 278us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:37:33.679659 (+467921us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 16:37:33.680005 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52964: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 16:37:33.680302 17434 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:33.208455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:43755 (local address 127.0.0.1:52964)
0926 16:37:33.208746 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:33.208758 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:33.208871 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:33.209181 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:33.209183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:33.209190 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:37:33.209275 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:33.209279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:33.210293 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:33.210295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:33.210916 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:33.210923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:33.211477 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:33.211480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:33.211544 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:33.211567 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:37:33.679909 (+468342us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 16:37:33.680127 (+ 218us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.158.190:43755: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":122,"threads_started":1}
W20260926 16:37:33.680301 17408 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52964: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260926 16:37:33.681048 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17338
2026-09-26T16:37:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (739 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:33 dist-test-slave-p39x krb5kdc[17446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:33 dist-test-slave-p39x krb5kdc[17446](info): set up 2 sockets
Sep 26 16:37:33 dist-test-slave-p39x krb5kdc[17446](info): commencing operation
krb5kdc: starting...
W20260926 16:37:34.989267 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:35 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:35.113750 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:43263
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:43263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:35.224624 17459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:35.224969 17459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:35.225044 17459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:35.229166 17459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:35.229275 17459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:35.229317 17459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:35.229367 17459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:35.229404 17459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:35.234843 17459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:43263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:43263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:35.235999 17459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:35.237097 17459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:35.245905 17464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:35.245884 17465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:35.245884 17467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:35.246282 17459 server_base.cc:1034] running on GCE node
I20260926 16:37:35.246914 17459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:35.248098 17459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:35.249285 17459 hybrid_clock.cc:648] HybridClock initialized: now 1790440655249253 us; error 48 us; skew 500 ppm
Sep 26 16:37:35 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440655, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:35.258424 17459 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:35.259505 17459 webserver.cc:460] Webserver started at http://127.11.158.190:40883/ using document root <none> and password file <none>
I20260926 16:37:35.260120 17459 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:35.260215 17459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:35.260509 17459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:35.262660 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "22a5fa82107d4d7f831de6fdc49936fb"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "859929e16f473c7fd5f06612aacb5cbf"
server_key_iv: "b769d90e2265e22f7d3ce46d5056c3df"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.263258 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "22a5fa82107d4d7f831de6fdc49936fb"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "859929e16f473c7fd5f06612aacb5cbf"
server_key_iv: "b769d90e2265e22f7d3ce46d5056c3df"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.267179 17459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:37:35.269976 17474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:35.271335 17459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:37:35.271490 17459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "22a5fa82107d4d7f831de6fdc49936fb"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "859929e16f473c7fd5f06612aacb5cbf"
server_key_iv: "b769d90e2265e22f7d3ce46d5056c3df"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.271646 17459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:35.299026 17459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:35.302196 17459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:35.302423 17459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:35.309572 17459 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:43263
I20260926 16:37:35.309597 17526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:43263 every 8 connection(s)
I20260926 16:37:35.310557 17459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:35.313452 17527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:35.319108 17527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb: Bootstrap starting.
I20260926 16:37:35.320751 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17459
I20260926 16:37:35.320901 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:35.321188 11898 external_mini_cluster.cc:1428] Setting key afb303cb456d1655ffda4c3880e17695
I20260926 16:37:35.322058 17527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:35.323055 17527 log.cc:824] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:35.325109 17527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb: No bootstrap required, opened a new log
I20260926 16:37:35.328730 17527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22a5fa82107d4d7f831de6fdc49936fb" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43263 } }
I20260926 16:37:35.328986 17527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:35.329067 17527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a5fa82107d4d7f831de6fdc49936fb, State: Initialized, Role: FOLLOWER
I20260926 16:37:35.329553 17527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22a5fa82107d4d7f831de6fdc49936fb" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43263 } }
I20260926 16:37:35.329716 17527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Sep 26 16:37:35 dist-test-slave-p39x krb5kdc[17446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:35.329941 17527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:35.330081 17527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:35.331267 17527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22a5fa82107d4d7f831de6fdc49936fb" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43263 } }
I20260926 16:37:35.331658 17527 leader_election.cc:302] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22a5fa82107d4d7f831de6fdc49936fb; no voters:
I20260926 16:37:35.332212 17527 leader_election.cc:288] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:35.332320 17532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:35.332564 17532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [term 1 LEADER]: Becoming Leader. State: Replica: 22a5fa82107d4d7f831de6fdc49936fb, State: Running, Role: LEADER
I20260926 16:37:35.332962 17532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22a5fa82107d4d7f831de6fdc49936fb" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43263 } }
I20260926 16:37:35.333727 17527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:35.335318 17534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22a5fa82107d4d7f831de6fdc49936fb. Latest consensus state: current_term: 1 leader_uuid: "22a5fa82107d4d7f831de6fdc49936fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22a5fa82107d4d7f831de6fdc49936fb" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43263 } } }
I20260926 16:37:35.335338 17533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22a5fa82107d4d7f831de6fdc49936fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22a5fa82107d4d7f831de6fdc49936fb" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43263 } } }
I20260926 16:37:35.335465 17534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:35.335466 17533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:35.336655 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:35.322456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.11.158.190:43263)
0926 16:37:35.322919 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:35.322934 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:35.322978 (+ 44us) server_negotiation.cc:407] Connection header received
0926 16:37:35.323698 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:35.323716 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:35.324048 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:35.324428 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:35.325302 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.326268 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.327141 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.327613 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.327777 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:35.330561 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:35.330568 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:35.330601 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:35.330629 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:35.332799 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:35.333344 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:35.333350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:35.333357 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:35.333500 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:35.333841 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:35.333844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:35.333846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:35.334444 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:35.334650 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:35.335297 (+ 647us) server_negotiation.cc:299] Negotiation successful
0926 16:37:35.335638 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":146,"threads_started":1}
I20260926 16:37:35.337675 17541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:35.341274 17541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:35.347115 17541 catalog_manager.cc:1347] Generated new cluster ID: 69be088df7c2418d958583d0a332feb4
I20260926 16:37:35.347203 17541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:35.370594 17541 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:35.371929 17541 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:35.377355 17541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22a5fa82107d4d7f831de6fdc49936fb: Generated new TSK 0
I20260926 16:37:35.378187 17541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:35.417685 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43263
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:35.530236 17553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:35.530501 17553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:35.530544 17553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:35.534775 17553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:35.534845 17553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:35.534905 17553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:35.540454 17553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:35.541575 17553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:35.542616 17553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:35.552769 17558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:35.552726 17559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:35.552781 17561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:35.552829 17553 server_base.cc:1034] running on GCE node
I20260926 16:37:35.553350 17553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:35.554159 17553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:35.555363 17553 hybrid_clock.cc:648] HybridClock initialized: now 1790440655555332 us; error 42 us; skew 500 ppm
Sep 26 16:37:35 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440655, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:35.558250 17553 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:35.559135 17553 webserver.cc:460] Webserver started at http://127.11.158.129:35859/ using document root <none> and password file <none>
I20260926 16:37:35.559696 17553 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:35.559759 17553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:35.559929 17553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:35.562093 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "987dd002d6174d2f95f59497689bb56e"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "1b8646056380a930279a96cfd6db8e94"
server_key_iv: "1a49e30c1757b907e3f40e7406e3c56c"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.562644 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "987dd002d6174d2f95f59497689bb56e"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "1b8646056380a930279a96cfd6db8e94"
server_key_iv: "1a49e30c1757b907e3f40e7406e3c56c"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.566531 17553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:37:35.569016 17568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:35.570261 17553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:35.570434 17553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "987dd002d6174d2f95f59497689bb56e"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "1b8646056380a930279a96cfd6db8e94"
server_key_iv: "1a49e30c1757b907e3f40e7406e3c56c"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.570559 17553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:35.590809 17553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:35.594079 17553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:35.594300 17553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:35.594935 17553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:35.595801 17553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:35.595847 17553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:35.595886 17553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:35.595938 17553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:35.604970 17553 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:34651
I20260926 16:37:35.605057 17681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:34651 every 8 connection(s)
I20260926 16:37:35.605924 17553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:35.615011 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17553
I20260926 16:37:35.615157 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
Sep 26 16:37:35 dist-test-slave-p39x krb5kdc[17446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440655, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:35.615447 11898 external_mini_cluster.cc:1428] Setting key 31ac6c2f49aa831a0db0bce5fcf1a4be
I20260926 16:37:35.619812 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:35.607883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:41173 (local address 127.11.158.190:43263)
0926 16:37:35.608033 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:35.608037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:35.608958 (+ 921us) server_negotiation.cc:407] Connection header received
0926 16:37:35.610252 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:35.610255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:35.610302 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:35.610377 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:35.611546 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.612063 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.613246 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.613467 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.613545 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:35.616021 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:35.616025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:35.616030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:35.616061 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:35.617643 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:35.618167 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:35.618173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:35.618177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:35.618284 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:35.618729 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:35.618733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:35.618737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:35.618981 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:35.619086 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:35.619582 (+ 496us) server_negotiation.cc:299] Negotiation successful
0926 16:37:35.619692 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:37:35.620708 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:35.608277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:43263 (local address 127.11.158.129:41173)
0926 16:37:35.608799 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:35.608842 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:35.610039 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:35.610545 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:35.610551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:35.610913 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:35.611414 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:35.611427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.612176 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.612179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:35.613129 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:35.613138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.613599 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.613602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:35.613746 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:35.614301 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:35.614317 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:35.615847 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:35.617776 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:35.617782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:35.617805 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:35.618060 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:35.618407 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:35.618412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:35.618416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:35.618611 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:35.619085 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:35.619090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:35.619339 (+ 249us) client_negotiation.cc:769] Sending connection context
0926 16:37:35.619502 (+ 163us) client_negotiation.cc:240] Negotiation successful
0926 16:37:35.619781 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":119,"threads_started":1}
I20260926 16:37:35.622020 17682 heartbeater.cc:347] Connected to a master server at 127.11.158.190:43263
I20260926 16:37:35.622273 17682 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:35.622782 17682 heartbeater.cc:511] Master 127.11.158.190:43263 requested a full tablet report, sending...
I20260926 16:37:35.624400 17491 ts_manager.cc:194] Registered new tserver with Master: 987dd002d6174d2f95f59497689bb56e (127.11.158.129:34651)
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:37:35.625629 17491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:41173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:35.647656 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43263
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:35.763427 17687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:35.763688 17687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:35.763731 17687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:35.767783 17687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:35.767854 17687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:35.767915 17687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:35.773337 17687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:35.774412 17687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:35.775488 17687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:35.784940 17692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:35.785081 17687 server_base.cc:1034] running on GCE node
W20260926 16:37:35.784971 17693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:35.784988 17695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:35.785610 17687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:35.786270 17687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:35.787458 17687 hybrid_clock.cc:648] HybridClock initialized: now 1790440655787442 us; error 27 us; skew 500 ppm
Sep 26 16:37:35 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440655, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:35.790138 17687 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:35.790992 17687 webserver.cc:460] Webserver started at http://127.11.158.130:42123/ using document root <none> and password file <none>
I20260926 16:37:35.791571 17687 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:35.791663 17687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:35.791913 17687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:35.793994 17687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "4940952368e14ba390fb72a0bb551f84"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "cc2be692777be3bc21019c064b421e05"
server_key_iv: "7b6737589c71c657fe34df961e7bbbe4"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.794559 17687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "4940952368e14ba390fb72a0bb551f84"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "cc2be692777be3bc21019c064b421e05"
server_key_iv: "7b6737589c71c657fe34df961e7bbbe4"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.798360 17687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 16:37:35.800907 17702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:35.802415 17687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:35.802587 17687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "4940952368e14ba390fb72a0bb551f84"
format_stamp: "Formatted at 2026-09-26 16:37:35 on dist-test-slave-p39x"
server_key: "cc2be692777be3bc21019c064b421e05"
server_key_iv: "7b6737589c71c657fe34df961e7bbbe4"
server_key_version: "encryptionkey@0"
I20260926 16:37:35.802731 17687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:35.830191 17687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:35.833693 17687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:35.833899 17687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:35.834578 17687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:35.835469 17687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:35.835543 17687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:35.835609 17687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:35.835661 17687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:35.846262 17687 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:38831
I20260926 16:37:35.846319 17815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:38831 every 8 connection(s)
I20260926 16:37:35.847255 17687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:35.855163 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17687
I20260926 16:37:35.855278 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:35.855556 11898 external_mini_cluster.cc:1428] Setting key e601ccb85d51c9960b2bb62c6168342f
Sep 26 16:37:35 dist-test-slave-p39x krb5kdc[17446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440655, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:35.861811 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:35.849197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:51275 (local address 127.11.158.190:43263)
0926 16:37:35.849345 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:35.849351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:35.850307 (+ 956us) server_negotiation.cc:407] Connection header received
0926 16:37:35.851627 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:35.851630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:35.851677 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:35.851752 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:35.853141 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.853683 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.854793 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.855005 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.855095 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:35.857933 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:35.857937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:35.857939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:35.857962 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:35.859478 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:35.860017 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:35.860024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:35.860028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:35.860124 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:35.860498 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:35.860504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:35.860508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:35.860777 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:35.860892 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:35.861453 (+ 561us) server_negotiation.cc:299] Negotiation successful
0926 16:37:35.861601 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:37:35.862548 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:35.849674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:43263 (local address 127.11.158.130:51275)
0926 16:37:35.850151 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:35.850195 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:35.851422 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:35.851925 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:35.851932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:35.852320 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:35.852977 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:35.852991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.853796 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.853798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:35.854668 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:35.854676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:35.855176 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:35.855182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:35.855386 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:35.856037 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:35.856055 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:35.857761 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:35.859593 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:35.859601 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:35.859623 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:35.859918 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:35.860215 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:35.860217 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:35.860219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:35.860407 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:35.860880 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:35.860884 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:35.861189 (+ 305us) client_negotiation.cc:769] Sending connection context
0926 16:37:35.861344 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 16:37:35.861634 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
I20260926 16:37:35.863703 17816 heartbeater.cc:347] Connected to a master server at 127.11.158.190:43263
I20260926 16:37:35.863960 17816 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:35.864509 17816 heartbeater.cc:511] Master 127.11.158.190:43263 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:37:35.865576 17491 ts_manager.cc:194] Registered new tserver with Master: 4940952368e14ba390fb72a0bb551f84 (127.11.158.130:38831)
I20260926 16:37:35.866129 17491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:51275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:35.883620 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43263
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:35.995383 17821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:35.995734 17821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:35.995808 17821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:35.999970 17821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:36.000069 17821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:36.000161 17821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:36.006386 17821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:36.007658 17821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:36.008762 17821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:36.019271 17827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:36.019265 17826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:36.020596 17821 server_base.cc:1034] running on GCE node
W20260926 16:37:36.023015 17829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:36.023501 17821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:36.024271 17821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:36.025486 17821 hybrid_clock.cc:648] HybridClock initialized: now 1790440656025464 us; error 33 us; skew 500 ppm
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.028209 17821 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:36.029192 17821 webserver.cc:460] Webserver started at http://127.11.158.131:40335/ using document root <none> and password file <none>
I20260926 16:37:36.029799 17821 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:36.029891 17821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:36.030123 17821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:36.032135 17821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "14da78b9172f42f98005a5b436ecc021"
format_stamp: "Formatted at 2026-09-26 16:37:36 on dist-test-slave-p39x"
server_key: "dbf9284fd362b7be080eaa1722e47598"
server_key_iv: "8d3f29c447ca13796a3123229d0198df"
server_key_version: "encryptionkey@0"
I20260926 16:37:36.032760 17821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "14da78b9172f42f98005a5b436ecc021"
format_stamp: "Formatted at 2026-09-26 16:37:36 on dist-test-slave-p39x"
server_key: "dbf9284fd362b7be080eaa1722e47598"
server_key_iv: "8d3f29c447ca13796a3123229d0198df"
server_key_version: "encryptionkey@0"
I20260926 16:37:36.036727 17821 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:37:36.039216 17836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:36.040611 17821 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:36.040748 17821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "14da78b9172f42f98005a5b436ecc021"
format_stamp: "Formatted at 2026-09-26 16:37:36 on dist-test-slave-p39x"
server_key: "dbf9284fd362b7be080eaa1722e47598"
server_key_iv: "8d3f29c447ca13796a3123229d0198df"
server_key_version: "encryptionkey@0"
I20260926 16:37:36.040865 17821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:36.060590 17821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:36.063452 17821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:36.063668 17821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:36.064265 17821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:36.065117 17821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:36.065188 17821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:36.065260 17821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:36.065311 17821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:36.074018 17821 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:32917
I20260926 16:37:36.074041 17949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:32917 every 8 connection(s)
I20260926 16:37:36.074963 17821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:36.080600 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17821
I20260926 16:37:36.080710 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:36.081029 11898 external_mini_cluster.cc:1428] Setting key f1d30265f9489d942224803d08ce5fb2
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:36.091075 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:36.076959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:58325 (local address 127.11.158.190:43263)
0926 16:37:36.077089 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:36.077093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:36.078031 (+ 938us) server_negotiation.cc:407] Connection header received
0926 16:37:36.079379 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:36.079382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:36.079442 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:36.079521 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:36.080840 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.081460 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.083230 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.083525 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.083638 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:36.087112 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:36.087116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:36.087118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:36.087159 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:36.088984 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.089538 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.089543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.089546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.089662 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.090028 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.090034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.090036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.090248 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:36.090331 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:36.090833 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 16:37:36.090948 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:36.091938 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:36.077410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:43263 (local address 127.11.158.131:58325)
0926 16:37:36.077884 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:36.077926 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:36.079171 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:36.079694 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:36.079701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:36.080056 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:36.080611 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:36.080625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.081608 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.081611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:36.083088 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:36.083098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.083684 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.083688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:36.083893 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:36.084740 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:36.084762 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:36.086921 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:36.089110 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:36.089117 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:36.089135 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:36.089404 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:36.089765 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:36.089768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:36.089769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:36.089921 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:36.090341 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:36.090346 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:36.090596 (+ 250us) client_negotiation.cc:769] Sending connection context
0926 16:37:36.090781 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 16:37:36.091052 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":114,"threads_started":1}
I20260926 16:37:36.093192 17950 heartbeater.cc:347] Connected to a master server at 127.11.158.190:43263
I20260926 16:37:36.093422 17950 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:36.093994 17950 heartbeater.cc:511] Master 127.11.158.190:43263 requested a full tablet report, sending...
I20260926 16:37:36.095234 17490 ts_manager.cc:194] Registered new tserver with Master: 14da78b9172f42f98005a5b436ecc021 (127.11.158.131:32917)
I20260926 16:37:36.095837 17490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:58325
I20260926 16:37:36.107672 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/test-admin.keytab.
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.124600 11898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 16:37:36.127369 17957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:36.125963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.11.158.129:34651)
0926 16:37:36.126263 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:36.126270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:36.126293 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:37:36.126343 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:36.126348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:36.126526 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:36.126646 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:36.127208 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47928: BlockingRecv error: recv got EOF from 127.0.0.1:47928 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
W20260926 16:37:36.127388 11898 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc' of type FILE
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.128794 11898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.129787 11898 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:36.140007 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:36.130671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.11.158.129:34651)
0926 16:37:36.130789 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:36.130793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:36.130942 (+ 149us) server_negotiation.cc:407] Connection header received
0926 16:37:36.131080 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:36.131099 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:36.131147 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:36.131234 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:36.131762 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.132619 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.133470 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.133843 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.133965 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:36.136469 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:36.136478 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:36.136491 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:36.136516 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:36.138315 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.138724 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.138729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.138731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.138809 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.139098 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.139100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.139101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.139333 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:36.139489 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:36.139660 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 16:37:36.139863 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260926 16:37:36.144387 17957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:36.142969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47934 (local address 127.11.158.129:34651)
0926 16:37:36.143239 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:36.143243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:36.143481 (+ 238us) server_negotiation.cc:407] Connection header received
0926 16:37:36.143636 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:36.143641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:36.143703 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:36.143792 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:36.144249 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47934: BlockingRecv error: recv got EOF from 127.0.0.1:47934 (error 108)
Metrics: {"server-negotiator.queue_time_us":154}
W20260926 16:37:36.144802 11898 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc)
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.145892 11898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.147008 11898 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:36.156306 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:36.147981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.11.158.129:34651)
0926 16:37:36.148143 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:36.148149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:36.148291 (+ 142us) server_negotiation.cc:407] Connection header received
0926 16:37:36.148540 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:36.148544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:36.148599 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:36.148696 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:36.149278 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.149934 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.150804 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.151049 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.151173 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:36.153493 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:36.153495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:36.153498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:36.153525 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:36.154758 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.155202 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.155204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.155205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.155276 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.155514 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.155516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.155517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.155706 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:36.155818 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:36.156088 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 16:37:36.156185 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 16:37:36.158886 17957 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:36.158065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47954 (local address 127.11.158.129:34651)
0926 16:37:36.158195 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:36.158199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:36.158268 (+ 69us) server_negotiation.cc:407] Connection header received
0926 16:37:36.158352 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:36.158354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:36.158388 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:36.158445 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:36.158778 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47954: BlockingRecv error: recv got EOF from 127.0.0.1:47954 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 16:37:36.158944 11898 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440622937842-11898-0/krb5kdc/krb5cc' of type FILE
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.159938 11898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:36.161031 11898 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:37:36.169435 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:36.161880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.11.158.129:34651)
0926 16:37:36.162005 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:36.162009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:36.162115 (+ 106us) server_negotiation.cc:407] Connection header received
0926 16:37:36.162230 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:36.162233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:36.162277 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:36.162358 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:36.162847 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.163335 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.164144 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:36.164441 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:36.164553 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:36.166733 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:36.166736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:36.166738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:36.166762 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:36.167956 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.168388 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.168391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.168393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.168472 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:36.168745 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:36.168748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:36.168749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:36.168909 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:36.169024 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:36.169226 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 16:37:36.169317 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:37:36.170557 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17553
I20260926 16:37:36.176192 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17687
I20260926 16:37:36.181340 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17821
I20260926 16:37:36.186642 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17459
2026-09-26T16:37:36Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2252 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17970](info): set up 2 sockets
Sep 26 16:37:36 dist-test-slave-p39x krb5kdc[17970](info): commencing operation
krb5kdc: starting...
W20260926 16:37:37.225577 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:37 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab.
Entry for principal oryx/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab.
I20260926 16:37:37.349920 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:32879
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab
--principal=oryx/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:32879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:37.460973 17983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:37.461333 17983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:37.461380 17983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:37.465688 17983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:37.465754 17983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:37.465775 17983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:37.465792 17983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:37.465808 17983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:37.471367 17983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:32879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab
--principal=oryx/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:32879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:37.472497 17983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:37.473483 17983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:37.482347 17991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:37.482406 17988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:37.482457 17983 server_base.cc:1034] running on GCE node
W20260926 16:37:37.482352 17989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:37.483245 17983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:37.484431 17983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:37.485659 17983 hybrid_clock.cc:648] HybridClock initialized: now 1790440657485610 us; error 65 us; skew 500 ppm
Sep 26 16:37:37 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:37.488596 17983 init.cc:377] Logged in from keytab as oryx/127.11.158.190@KRBTEST.COM (short username oryx)
I20260926 16:37:37.489565 17983 webserver.cc:460] Webserver started at http://127.11.158.190:42685/ using document root <none> and password file <none>
I20260926 16:37:37.490165 17983 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:37.490258 17983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:37.490505 17983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:37.492977 17983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "32618b75554bdb5d833fe7b335f22cd5"
server_key_iv: "a0020c0cba18f213ea487958e45e4869"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.493599 17983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "32618b75554bdb5d833fe7b335f22cd5"
server_key_iv: "a0020c0cba18f213ea487958e45e4869"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.497817 17983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:37:37.500579 17998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:37.502739 17983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:37:37.502933 17983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "32618b75554bdb5d833fe7b335f22cd5"
server_key_iv: "a0020c0cba18f213ea487958e45e4869"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.503086 17983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:37.524256 17983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:37.527638 17983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:37.527868 17983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:37.535135 17983 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:32879
I20260926 16:37:37.535420 18050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:32879 every 8 connection(s)
I20260926 16:37:37.536227 17983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:37.536487 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 17983
I20260926 16:37:37.536631 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:37.537011 11898 external_mini_cluster.cc:1428] Setting key 184ba15f7f61f177a915cd991fd806ff
I20260926 16:37:37.539253 18051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:37.545889 18051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: Bootstrap starting.
Sep 26 16:37:37 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:37.548631 18051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:37.549522 18051 log.cc:824] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:37.551635 18051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: No bootstrap required, opened a new log
I20260926 16:37:37.552018 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:37.538529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.11.158.190:32879)
0926 16:37:37.538938 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:37.538953 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:37.538989 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:37:37.539691 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:37.539709 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:37.540044 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:37.540501 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:37.541280 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:37.542307 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:37.543414 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:37.543886 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:37.544004 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:37.546830 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:37.546837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:37.546852 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:37.546883 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:37.548991 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:37.549438 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:37.549443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:37.549449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:37.549587 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:37.549979 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:37.549983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:37.549985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:37.550469 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:37.550622 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:37.550941 (+ 319us) server_negotiation.cc:299] Negotiation successful
0926 16:37:37.551267 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":136,"threads_started":1}
I20260926 16:37:37.555121 18051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } }
I20260926 16:37:37.555361 18051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:37.555440 18051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5b1419a34348c4814c9e5f9e9bcbf8, State: Initialized, Role: FOLLOWER
I20260926 16:37:37.555852 18051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } }
I20260926 16:37:37.555998 18051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:37.556089 18051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:37.556205 18051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:37.557156 18051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } }
I20260926 16:37:37.557530 18051 leader_election.cc:302] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b5b1419a34348c4814c9e5f9e9bcbf8; no voters:
I20260926 16:37:37.557890 18051 leader_election.cc:288] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:37.558033 18056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:37.558290 18056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 1 LEADER]: Becoming Leader. State: Replica: 9b5b1419a34348c4814c9e5f9e9bcbf8, State: Running, Role: LEADER
I20260926 16:37:37.558602 18056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } }
I20260926 16:37:37.559108 18051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:37.559657 18056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b5b1419a34348c4814c9e5f9e9bcbf8. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } }
I20260926 16:37:37.559865 18056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:37.560079 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } }
I20260926 16:37:37.560173 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:37.560292 18064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:37.563531 18064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:37.569110 18064 catalog_manager.cc:1347] Generated new cluster ID: 0d6648a44d70470890f55acda11ed1ab
I20260926 16:37:37.569180 18064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:37.579135 18064 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:37.580050 18064 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:37.584316 18064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: Generated new TSK 0
I20260926 16:37:37.584985 18064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab.
Entry for principal oryx/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab.
I20260926 16:37:37.623831 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:32879
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab
--principal=oryx/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:37.741634 18077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:37.741900 18077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:37.741979 18077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:37.746031 18077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:37.746091 18077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:37.746150 18077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:37.751446 18077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab
--principal=oryx/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:32879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:37.752547 18077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:37.753577 18077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:37.763456 18082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:37.763546 18083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:37.763473 18085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:37.763522 18077 server_base.cc:1034] running on GCE node
I20260926 16:37:37.764202 18077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:37.764902 18077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:37.766114 18077 hybrid_clock.cc:648] HybridClock initialized: now 1790440657766095 us; error 32 us; skew 500 ppm
Sep 26 16:37:37 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:37.768816 18077 init.cc:377] Logged in from keytab as oryx/127.11.158.129@KRBTEST.COM (short username oryx)
I20260926 16:37:37.769666 18077 webserver.cc:460] Webserver started at http://127.11.158.129:44591/ using document root <none> and password file <none>
I20260926 16:37:37.770327 18077 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:37.770414 18077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:37.770645 18077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:37.772742 18077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "676b8797be7847339c9336fe0e2b191f"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "374e8cecb7adb6d29af2bbeefa5f0716"
server_key_iv: "fe1e903185df6a0a5c54b8a661dc65a4"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.773285 18077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "676b8797be7847339c9336fe0e2b191f"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "374e8cecb7adb6d29af2bbeefa5f0716"
server_key_iv: "fe1e903185df6a0a5c54b8a661dc65a4"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.776901 18077 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:37.779412 18092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:37.780829 18077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:37.780961 18077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "676b8797be7847339c9336fe0e2b191f"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "374e8cecb7adb6d29af2bbeefa5f0716"
server_key_iv: "fe1e903185df6a0a5c54b8a661dc65a4"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.781076 18077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:37.791968 18077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:37.794886 18077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:37.795085 18077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:37.795661 18077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:37.796522 18077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:37.796589 18077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:37.796669 18077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:37.796701 18077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:37.805667 18077 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:43867
I20260926 16:37:37.805686 18205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:43867 every 8 connection(s)
I20260926 16:37:37.806635 18077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:37.810742 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 18077
I20260926 16:37:37.810889 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:37.811175 11898 external_mini_cluster.cc:1428] Setting key 1d64a6c69d879cf8b0d891c4d0752d3c
Sep 26 16:37:37 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:37:37.821334 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:37.808530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:46763 (local address 127.11.158.190:32879)
0926 16:37:37.808680 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:37.808684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:37.809538 (+ 854us) server_negotiation.cc:407] Connection header received
0926 16:37:37.810897 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:37.810900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:37.810966 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:37.811043 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:37.812576 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:37.813349 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:37.814592 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:37.814850 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:37.814976 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:37.817655 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:37.817658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:37.817660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:37.817684 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:37.819191 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:37.819708 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:37.819716 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:37.819718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:37.819790 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:37.820140 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:37.820143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:37.820144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:37.820322 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:37.820465 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:37.821106 (+ 641us) server_negotiation.cc:299] Negotiation successful
0926 16:37:37.821212 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:37:37.822137 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:37.808895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.129:46763)
0926 16:37:37.809353 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:37.809405 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:37.810684 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:37.811214 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:37.811222 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:37.811725 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:37.812399 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:37.812420 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:37.813510 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:37.813513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:37.814462 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:37.814472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:37.815001 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:37.815005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:37.815191 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:37.815807 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:37.815823 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:37.817473 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:37.819314 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:37.819320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:37.819335 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:37.819595 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:37.819883 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:37.819887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:37.819889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:37.820030 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:37.820482 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:37.820491 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:37.820833 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 16:37:37.821026 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 16:37:37.821292 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.130@KRBTEST.COM" created.
I20260926 16:37:37.823436 18206 heartbeater.cc:347] Connected to a master server at 127.11.158.190:32879
I20260926 16:37:37.823670 18206 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:37.824158 18206 heartbeater.cc:511] Master 127.11.158.190:32879 requested a full tablet report, sending...
I20260926 16:37:37.825780 18015 ts_manager.cc:194] Registered new tserver with Master: 676b8797be7847339c9336fe0e2b191f (127.11.158.129:43867)
I20260926 16:37:37.826989 18015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:46763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab.
Entry for principal oryx/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab.
I20260926 16:37:37.841295 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:32879
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab
--principal=oryx/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:37.951417 18211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:37.951706 18211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:37.951782 18211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:37.956568 18211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:37.956655 18211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:37.956771 18211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:37.962383 18211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab
--principal=oryx/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:32879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:37.963397 18211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:37.964447 18211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:37.974210 18217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:37.974247 18219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:37.974213 18216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:37.974505 18211 server_base.cc:1034] running on GCE node
I20260926 16:37:37.974952 18211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:37.975580 18211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:37.976778 18211 hybrid_clock.cc:648] HybridClock initialized: now 1790440657976768 us; error 31 us; skew 500 ppm
Sep 26 16:37:37 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:37.979641 18211 init.cc:377] Logged in from keytab as oryx/127.11.158.130@KRBTEST.COM (short username oryx)
I20260926 16:37:37.980645 18211 webserver.cc:460] Webserver started at http://127.11.158.130:46493/ using document root <none> and password file <none>
I20260926 16:37:37.981235 18211 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:37.981328 18211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:37.981564 18211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:37.983641 18211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "5d5331360dd240e09eb6b57453ede29b"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "17e812281e1b2806a4398812e8ebdf2e"
server_key_iv: "7b2f7d6eafb3a36bbf9abf63589b5ded"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.984218 18211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "5d5331360dd240e09eb6b57453ede29b"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "17e812281e1b2806a4398812e8ebdf2e"
server_key_iv: "7b2f7d6eafb3a36bbf9abf63589b5ded"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.988042 18211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:37.990653 18226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:37.991909 18211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:37.992050 18211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "5d5331360dd240e09eb6b57453ede29b"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "17e812281e1b2806a4398812e8ebdf2e"
server_key_iv: "7b2f7d6eafb3a36bbf9abf63589b5ded"
server_key_version: "encryptionkey@0"
I20260926 16:37:37.992166 18211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:38.011674 18211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:38.015107 18211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:38.015329 18211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:38.015942 18211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:38.016852 18211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:38.016927 18211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.017000 18211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:38.017045 18211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.026059 18211 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:46573
I20260926 16:37:38.026090 18339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:46573 every 8 connection(s)
I20260926 16:37:38.026988 18211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:38.027989 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 18211
I20260926 16:37:38.028110 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:38.028486 11898 external_mini_cluster.cc:1428] Setting key 3dc238023431022c8e13a238c2c1f504
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.131@KRBTEST.COM" created.
I20260926 16:37:38.043125 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.029123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:37587 (local address 127.11.158.190:32879)
0926 16:37:38.029467 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.029471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.030984 (+ 1513us) server_negotiation.cc:407] Connection header received
0926 16:37:38.032380 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.032385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.032449 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.032539 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.033693 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.034643 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.035820 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.036073 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.036200 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.038997 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.039005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.039007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.039033 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.040689 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.041296 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.041300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.041302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.041386 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.041739 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.041742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.041744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.041997 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.042098 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.042755 (+ 657us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.042892 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260926 16:37:38.044349 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.029650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:37587)
0926 16:37:38.030838 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.030882 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.032140 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.032689 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.032696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.033054 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.033527 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.033541 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.034784 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.034787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.035702 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.035711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.036239 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.036243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.036486 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.037103 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.037120 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.038817 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.040815 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.040822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.040838 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.041185 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.041478 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.041482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.041484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.041631 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.042126 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.042131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.042457 (+ 326us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.042709 (+ 252us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.043111 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":125,"threads_started":1}
I20260926 16:37:38.045601 18340 heartbeater.cc:347] Connected to a master server at 127.11.158.190:32879
I20260926 16:37:38.045867 18340 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:38.046367 18340 heartbeater.cc:511] Master 127.11.158.190:32879 requested a full tablet report, sending...
I20260926 16:37:38.047441 18015 ts_manager.cc:194] Registered new tserver with Master: 5d5331360dd240e09eb6b57453ede29b (127.11.158.130:46573)
I20260926 16:37:38.048027 18015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:37587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab.
Entry for principal oryx/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab.
I20260926 16:37:38.059724 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:32879
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab
--principal=oryx/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:38.175043 18345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:38.175302 18345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:38.175345 18345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:38.179468 18345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:38.179539 18345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:38.179600 18345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:38.185613 18345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab
--principal=oryx/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:32879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:38.186677 18345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:38.187695 18345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:38.197304 18350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:38.197304 18351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:38.197547 18353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:38.197745 18345 server_base.cc:1034] running on GCE node
I20260926 16:37:38.198207 18345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:38.198920 18345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:38.200109 18345 hybrid_clock.cc:648] HybridClock initialized: now 1790440658200092 us; error 30 us; skew 500 ppm
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:38.204169 18345 init.cc:377] Logged in from keytab as oryx/127.11.158.131@KRBTEST.COM (short username oryx)
I20260926 16:37:38.205240 18345 webserver.cc:460] Webserver started at http://127.11.158.131:38769/ using document root <none> and password file <none>
I20260926 16:37:38.205829 18345 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:38.205894 18345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:38.206141 18345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:38.208552 18345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "f1bf0a9157a74f05a449cd80a6bf4af4"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "124622dc045f85eb45f687fd29bc192b"
server_key_iv: "ec6a5531e68cc7e9d8e0fab8988d8474"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.209152 18345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "f1bf0a9157a74f05a449cd80a6bf4af4"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "124622dc045f85eb45f687fd29bc192b"
server_key_iv: "ec6a5531e68cc7e9d8e0fab8988d8474"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.213063 18345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:37:38.215478 18360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.216913 18345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:38.217057 18345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "f1bf0a9157a74f05a449cd80a6bf4af4"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "124622dc045f85eb45f687fd29bc192b"
server_key_iv: "ec6a5531e68cc7e9d8e0fab8988d8474"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.217177 18345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:38.231038 18345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:38.234479 18345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:38.234668 18345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:38.235332 18345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:38.236174 18345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:38.236224 18345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.236296 18345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:38.236402 18345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260926 16:37:38.245656 18345 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:40297
I20260926 16:37:38.245674 18473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:40297 every 8 connection(s)
I20260926 16:37:38.246600 18345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:38.256631 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 18345
I20260926 16:37:38.256760 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:38.257085 11898 external_mini_cluster.cc:1428] Setting key 386c08f62e75afc16fdcadd703963301
I20260926 16:37:38.260924 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.248426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:58927 (local address 127.11.158.190:32879)
0926 16:37:38.248556 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.248561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.249414 (+ 853us) server_negotiation.cc:407] Connection header received
0926 16:37:38.250800 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.250804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.250862 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.250945 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.252292 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.252866 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.253997 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.254234 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.254349 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.256930 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.256933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.256935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.256961 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.258756 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.259272 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.259277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.259278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.259366 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.259752 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.259758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.259759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.259951 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.260057 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.260637 (+ 580us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.260764 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:37:38.261758 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.248815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.131:58927)
0926 16:37:38.249264 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.249305 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.250588 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.251123 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.251130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.251617 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.252131 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.252146 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.252997 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.253001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.253870 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.253879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.254390 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.254394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.254560 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.255216 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.255232 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.256744 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.258871 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.258878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.258894 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.259165 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.259456 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.259462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.259503 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.259654 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.260055 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.260062 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.260316 (+ 254us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.260592 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.260926 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
I20260926 16:37:38.262836 18474 heartbeater.cc:347] Connected to a master server at 127.11.158.190:32879
I20260926 16:37:38.263095 18474 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:38.263592 18474 heartbeater.cc:511] Master 127.11.158.190:32879 requested a full tablet report, sending...
I20260926 16:37:38.264743 18015 ts_manager.cc:194] Registered new tserver with Master: f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297)
I20260926 16:37:38.265347 18015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:58927
I20260926 16:37:38.271318 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.158.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.189.keytab.
Entry for principal oryx/127.11.158.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.189.keytab.
I20260926 16:37:38.300580 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:46079
--webserver_interface=127.11.158.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.189.keytab
--principal=oryx/127.11.158.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:32879,127.11.158.189:46079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:38.417373 18479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:38.417701 18479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:38.417793 18479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:38.422062 18479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:38.422145 18479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:38.422183 18479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:38.422237 18479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:38.422273 18479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:38.427737 18479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:32879,127.11.158.189:46079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.189.keytab
--principal=oryx/127.11.158.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:46079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.158.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
W20260926 16:37:38.428018 18479 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.158.190:32879,127.11.158.189:46079'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 16:37:38.428992 18479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:38.430042 18479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:38.438825 18484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:38.438825 18485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:38.438925 18487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:38.439188 18479 server_base.cc:1034] running on GCE node
I20260926 16:37:38.439797 18479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:38.441033 18479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:38.442203 18479 hybrid_clock.cc:648] HybridClock initialized: now 1790440658442176 us; error 42 us; skew 500 ppm
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:38.445129 18479 init.cc:377] Logged in from keytab as oryx/127.11.158.189@KRBTEST.COM (short username oryx)
I20260926 16:37:38.445951 18479 webserver.cc:460] Webserver started at http://127.11.158.189:40203/ using document root <none> and password file <none>
I20260926 16:37:38.446456 18479 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:38.446537 18479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:38.446761 18479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:38.448828 18479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/instance:
uuid: "0e8c28627ef94c9d824564d7b1fba272"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "deb7da35344de54bfed81808a1c69310"
server_key_iv: "a533f39ea87d6de4f5f4457758aacc55"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.449371 18479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal/instance:
uuid: "0e8c28627ef94c9d824564d7b1fba272"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "deb7da35344de54bfed81808a1c69310"
server_key_iv: "a533f39ea87d6de4f5f4457758aacc55"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.453203 18479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:37:38.455911 18494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.457480 18479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:37:38.457661 18479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
uuid: "0e8c28627ef94c9d824564d7b1fba272"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "deb7da35344de54bfed81808a1c69310"
server_key_iv: "a533f39ea87d6de4f5f4457758aacc55"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.457798 18479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:38.481422 18479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:38.484643 18479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:38.484877 18479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:38.491986 18479 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.189:46079
I20260926 16:37:38.492290 18546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.189:46079 every 8 connection(s)
I20260926 16:37:38.493142 18479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
I20260926 16:37:38.495949 18547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:38.497637 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 18479
I20260926 16:37:38.500437 18547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.158.189@KRBTEST.COM
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.189@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:38.513041 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.502149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56186 (local address 127.11.158.190:32879)
0926 16:37:38.502286 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.502291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.502656 (+ 365us) server_negotiation.cc:407] Connection header received
0926 16:37:38.502769 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.502772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.502937 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.503067 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.503873 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.504494 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.505504 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.505755 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.505879 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.509035 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.509041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.509046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.509084 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.510634 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.511458 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.511466 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.511470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.511564 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.511865 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.511871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.511872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.512118 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.512256 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.512776 (+ 520us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.512888 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:37:38.513741 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.502100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:56186)
0926 16:37:38.502502 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.502549 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.502669 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.503153 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.503163 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.503538 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.503712 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.503723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.504643 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.504646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.505365 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.505374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.505914 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.505919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.506036 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.506668 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.506685 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.508861 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.511081 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.511091 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.511098 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.511333 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.511672 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.511677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.511679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.511766 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.512252 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.512260 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.512453 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.512691 (+ 238us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.512875 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":133,"threads_started":1}
I20260926 16:37:38.513742 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.499152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.11.158.189:46079)
0926 16:37:38.499618 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.499635 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.499670 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:37:38.500268 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.500284 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.500700 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.501066 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.502834 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.503796 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.504707 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.505201 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.505383 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.508529 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.508542 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.508553 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.508576 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.510619 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.511065 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.511070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.511075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.511184 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.511483 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.511486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.511488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.511906 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.512128 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.512465 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.512807 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":140,"threads_started":1}
I20260926 16:37:38.515234 18547 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.189@KRBTEST.COM for oryx/127.11.158.189@KRBTEST.COM
I20260926 16:37:38.523780 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.515766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.0.0.1:42182)
0926 16:37:38.515927 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.515941 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.516054 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.516310 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.516313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.516514 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.516625 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.516631 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.517780 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.517786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.518351 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.518358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.518834 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.518838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.518935 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.519466 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.519479 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.520876 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.522467 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.522470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.522471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.522689 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.522926 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.522929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.522930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.523010 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.523329 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.523332 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.523410 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.523495 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.523618 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260926 16:37:38.523806 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.515795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42182 (local address 127.11.158.189:46079)
0926 16:37:38.515900 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.515905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.516013 (+ 108us) server_negotiation.cc:407] Connection header received
0926 16:37:38.516157 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.516161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.516206 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.516300 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.516742 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.517664 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.518470 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.518723 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.518839 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.521000 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.521003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.521005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.521025 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.522371 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.522771 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.522774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.522775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.522842 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.523089 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.523091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.523092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.523249 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.523320 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.523542 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.523685 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":33}
I20260926 16:37:38.526926 18547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Bootstrap starting.
I20260926 16:37:38.529412 18547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:38.530194 18547 log.cc:824] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:38.532173 18547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: No bootstrap required, opened a new log
I20260926 16:37:38.535445 18547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } }
I20260926 16:37:38.535684 18547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:38.535789 18547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8c28627ef94c9d824564d7b1fba272, State: Initialized, Role: FOLLOWER
I20260926 16:37:38.536260 18547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } }
I20260926 16:37:38.536826 18553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } } }
I20260926 16:37:38.536959 18553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:37:38.537652 18547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 16:37:38.542471 18564 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:37:38.542616 18564 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:37:38.544557 18479 master_runner.cc:428] Detected that this master 0e8c28627ef94c9d824564d7b1fba272 is joining an existing cluster
I20260926 16:37:38.544633 18479 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.158.189:46079
I20260926 16:37:38.556913 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.547733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.11.158.189:46079)
0926 16:37:38.547843 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.547848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.547998 (+ 150us) server_negotiation.cc:407] Connection header received
0926 16:37:38.548177 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.548180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.548215 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.548257 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.548905 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.549791 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.551039 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.551420 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.551544 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.553054 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.553059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.553062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.553098 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.554934 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.555366 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.555370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.555371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.555452 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.555717 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.555723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.555726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.555897 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.556008 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.556548 (+ 540us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.556733 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":40}
I20260926 16:37:38.556926 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.547642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.0.0.1:42198)
0926 16:37:38.547928 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.547940 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.548073 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.548400 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.548403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.548678 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.548802 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.548809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.549916 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.549919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.550902 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.550912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.551547 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.551550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.551647 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.552458 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.552475 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.552915 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.555044 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.555049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.555055 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.555248 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.555542 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.555547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.555549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.555622 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.556258 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.556259 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.556389 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.556506 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.556685 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":23,"spinlock_wait_cycles":19584,"thread_start_us":69,"threads_started":1}
I20260926 16:37:38.557737 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.547350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56188 (local address 127.11.158.190:32879)
0926 16:37:38.547487 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.547493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.547963 (+ 470us) server_negotiation.cc:407] Connection header received
0926 16:37:38.548214 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.548218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.548272 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.548361 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.550145 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.550932 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.551898 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.552380 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.552494 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.554043 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.554049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.554054 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.554090 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.555874 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.556275 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.556278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.556281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.556407 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.556749 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.556752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.556754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.556981 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.557058 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.557442 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.557571 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:37:38.558336 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.547261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:56188)
0926 16:37:38.547872 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.547888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.548073 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.548535 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.548538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.548837 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.550010 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.550019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.551053 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.551056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.551759 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.551769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.552510 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.552515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.552625 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.553402 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.553418 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.553894 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.555989 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.555992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.555994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.556178 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.556527 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.556530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.556532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.556631 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.557147 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.557150 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.557242 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.557952 (+ 710us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.558125 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":5,"spinlock_wait_cycles":14464,"thread_start_us":95,"threads_started":1}
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.190@KRBTEST.COM for oryx/127.11.158.189@KRBTEST.COM
I20260926 16:37:38.573071 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.562945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.11.158.189:46079)
0926 16:37:38.563058 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.563064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.563394 (+ 330us) server_negotiation.cc:407] Connection header received
0926 16:37:38.563598 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.563601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.563658 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.563749 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.564449 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.564959 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.566043 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.566332 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.566432 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.569010 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.569016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.569019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.569053 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.570870 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.571473 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.571479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.571484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.571580 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.571915 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.571918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.571920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.572162 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.572276 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.572758 (+ 482us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.572885 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:37:38.573203 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.562920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.0.0.1:42202)
0926 16:37:38.563277 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.563317 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.563465 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.563785 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.563791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.564179 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.564288 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.564294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.565204 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.565211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.565898 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.565908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.566580 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.566583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.566706 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.567315 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.567327 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.568810 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.570995 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.571006 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.571013 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.571355 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.571696 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.571699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.571702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.571803 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.572306 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.572313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.572504 (+ 191us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.572691 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.572949 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":105,"threads_started":1}
I20260926 16:37:38.574860 18015 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.158.189:46079: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272"
member_type: NON_VOTER
last_known_addr {
host: "127.11.158.189"
port: 46079
}
attrs {
promote: true
}
}
dest_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8"
cas_config_opid_index: -1
I20260926 16:37:38.575523 18015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } }
I20260926 16:37:38.577322 18574 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0e8c28627ef94c9d824564d7b1fba272 (127.11.158.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } }
I20260926 16:37:38.577956 18521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.11.158.188@KRBTEST.COM; defaulting to no policy
I20260926 16:37:38.579381 18521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Refusing update from remote peer 9b5b1419a34348c4814c9e5f9e9bcbf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 16:37:38.579732 18553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:38.579878 18553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } }
I20260926 16:37:38.580292 18574 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.158.189:46079
I20260926 16:37:38.580181 18575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:38.580829 18479 master.cc:537] Master@127.11.158.189:46079 shutting down...
I20260926 16:37:38.581063 18553 leader_election.cc:288] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.188@KRBTEST.COM" created.
I20260926 16:37:38.581952 18025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e8c28627ef94c9d824564d7b1fba272" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" is_pre_election: true
I20260926 16:37:38.582545 18497 leader_election.cc:302] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 0e8c28627ef94c9d824564d7b1fba272; no voters: 9b5b1419a34348c4814c9e5f9e9bcbf8
I20260926 16:37:38.582854 18576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } }
I20260926 16:37:38.583063 18553 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:37:38.583171 18576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
W20260926 16:37:38.584736 18001 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 16:37:38.585790 18001 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 -> Peer 0e8c28627ef94c9d824564d7b1fba272 (127.11.158.189:46079): Couldn't send request to peer 0e8c28627ef94c9d824564d7b1fba272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 16:37:38.587778 18479 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:38.588081 18479 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:38.588217 18479 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: stopping tablet replica
I20260926 16:37:38.604272 18479 master.cc:552] Master@127.11.158.189:46079 shutdown complete.
I20260926 16:37:38.604406 18479 master_runner.cc:305] Clearing existing system tablet
I20260926 16:37:38.604787 18479 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:38.604859 18479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:38.606465 18479 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:38.607949 18582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.608204 18479 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:38.608376 18479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 16:37:38.608464 18479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
uuid: "0e8c28627ef94c9d824564d7b1fba272"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "deb7da35344de54bfed81808a1c69310"
server_key_iv: "a533f39ea87d6de4f5f4457758aacc55"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.188.keytab.
Entry for principal oryx/127.11.158.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.188.keytab.
I20260926 16:37:38.609653 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:45501
--webserver_interface=127.11.158.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.188.keytab
--principal=oryx/127.11.158.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:37:38.611021 18479 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:38.613162 18479 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 16:37:38.613278 18479 log.cc:1196] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 16:37:38.613516 18479 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Deleting consensus metadata
I20260926 16:37:38.613727 18479 master_runner.cc:315] Copying system tablet from 127.11.158.190:32879
I20260926 16:37:38.614920 18479 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: tablet copy: Beginning tablet copy session from remote peer at address 127.11.158.190:32879
I20260926 16:37:38.623106 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.615592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.11.158.190:32879)
0926 16:37:38.615742 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.615747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.615935 (+ 188us) server_negotiation.cc:407] Connection header received
0926 16:37:38.616086 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.616089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.616145 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.616234 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.616797 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.617336 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.618188 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.618429 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.618541 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.619708 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.619712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.619715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.619745 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.621349 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.621848 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.621851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.621853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.621947 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.622242 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.622247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.622249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.622443 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.622547 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.622826 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.622942 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 16:37:38.623116 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.615524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:56196)
0926 16:37:38.615837 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.615850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.615987 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.616251 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.616254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.616554 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.616668 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.616674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.617451 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.617457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.618054 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.618062 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.618546 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.618550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.618644 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.619211 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.619223 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.619553 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.621466 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.621470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.621472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.621723 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.622042 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.622046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.622048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.622146 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.622547 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.622550 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.622655 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.622767 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.622927 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":87,"threads_started":1}
I20260926 16:37:38.624099 18035 tablet_copy_service.cc:140] P 9b5b1419a34348c4814c9e5f9e9bcbf8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0e8c28627ef94c9d824564d7b1fba272 ({username='oryx', principal='oryx/127.11.158.189@KRBTEST.COM'} at 127.0.0.1:56196)
I20260926 16:37:38.624265 18035 tablet_copy_service.cc:161] P 9b5b1419a34348c4814c9e5f9e9bcbf8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0e8c28627ef94c9d824564d7b1fba272 at {username='oryx', principal='oryx/127.11.158.189@KRBTEST.COM'} at 127.0.0.1:56196: session id = 0e8c28627ef94c9d824564d7b1fba272-00000000000000000000000000000000
I20260926 16:37:38.625788 18035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 16:37:38.627362 18479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 16:37:38.629407 18479 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: tablet copy: Starting download of 0 data blocks...
I20260926 16:37:38.629619 18479 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: tablet copy: Starting download of 1 WAL segments...
I20260926 16:37:38.631392 18479 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 16:37:38.633141 18035 tablet_copy_service.cc:342] P 9b5b1419a34348c4814c9e5f9e9bcbf8: Request end of tablet copy session 0e8c28627ef94c9d824564d7b1fba272-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.158.189@KRBTEST.COM'} at 127.0.0.1:56196
I20260926 16:37:38.633261 18035 tablet_copy_service.cc:433] P 9b5b1419a34348c4814c9e5f9e9bcbf8: ending tablet copy session 0e8c28627ef94c9d824564d7b1fba272-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0e8c28627ef94c9d824564d7b1fba272
I20260926 16:37:38.640225 18479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:38.642117 18590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:38.642261 18593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:38.642468 18479 server_base.cc:1034] running on GCE node
W20260926 16:37:38.644516 18591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:38.644893 18479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:38.645262 18479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:38.646407 18479 hybrid_clock.cc:648] HybridClock initialized: now 1790440658646394 us; error 33 us; skew 500 ppm
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:38.647961 18479 init.cc:377] Logged in from keytab as oryx/127.11.158.189@KRBTEST.COM (short username oryx)
I20260926 16:37:38.648543 18479 webserver.cc:460] Webserver started at http://127.11.158.189:46859/ using document root <none> and password file <none>
I20260926 16:37:38.648765 18479 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:38.648820 18479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:38.649973 18479 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:38.650929 18600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.651177 18479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:38.651248 18479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
uuid: "0e8c28627ef94c9d824564d7b1fba272"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "deb7da35344de54bfed81808a1c69310"
server_key_iv: "a533f39ea87d6de4f5f4457758aacc55"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.651350 18479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:38.665441 18479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:38.667484 18479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:38.672385 18479 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.189:46079
I20260926 16:37:38.673059 18479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
I20260926 16:37:38.674579 18653 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:37:38.675503 18653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Bootstrap starting.
I20260926 16:37:38.683835 18653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:37:38.684171 18652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.189:46079 every 8 connection(s)
I20260926 16:37:38.684201 18653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Bootstrap complete.
I20260926 16:37:38.684904 18653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } }
I20260926 16:37:38.685109 18653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0e8c28627ef94c9d824564d7b1fba272, State: Initialized, Role: LEARNER
I20260926 16:37:38.685228 18653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } }
I20260926 16:37:38.685734 18653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:38.688385 18656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } }
I20260926 16:37:38.688716 18656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:37:38.689368 18667 catalog_manager.cc:1259] Loaded cluster ID: 0d6648a44d70470890f55acda11ed1ab
I20260926 16:37:38.689425 18667 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: loading cluster ID for follower catalog manager: success
I20260926 16:37:38.691529 18667 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: acquiring CA information for follower catalog manager: success
I20260926 16:37:38.693048 18667 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 16:37:38.754071 18583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:38.754487 18583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:38.754572 18583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:38.759210 18583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:38.759279 18583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:38.759298 18583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:38.759316 18583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:38.759331 18583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:38.765055 18583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.188.keytab
--principal=oryx/127.11.158.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:45501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.158.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:38.766152 18583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:38.767242 18583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:38.776523 18672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:38.776597 18673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:38.776523 18675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:38.776738 18583 server_base.cc:1034] running on GCE node
I20260926 16:37:38.777348 18583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:38.778517 18583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:38.779726 18583 hybrid_clock.cc:648] HybridClock initialized: now 1790440658779703 us; error 43 us; skew 500 ppm
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:38.782649 18583 init.cc:377] Logged in from keytab as oryx/127.11.158.188@KRBTEST.COM (short username oryx)
I20260926 16:37:38.783528 18583 webserver.cc:460] Webserver started at http://127.11.158.188:43551/ using document root <none> and password file <none>
I20260926 16:37:38.784081 18583 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:38.784137 18583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:38.784303 18583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:38.786943 18583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/instance:
uuid: "78cfcea6928e460485d6300e5b396d80"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "a207fb6e213603e3f2c3e8bc7fb26dd5"
server_key_iv: "babc29a2d45d1a1e58553118b7bc24ab"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.787554 18583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal/instance:
uuid: "78cfcea6928e460485d6300e5b396d80"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "a207fb6e213603e3f2c3e8bc7fb26dd5"
server_key_iv: "babc29a2d45d1a1e58553118b7bc24ab"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.791427 18583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:37:38.794121 18682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.795608 18583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:38.795759 18583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
uuid: "78cfcea6928e460485d6300e5b396d80"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "a207fb6e213603e3f2c3e8bc7fb26dd5"
server_key_iv: "babc29a2d45d1a1e58553118b7bc24ab"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.795859 18583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:38.825750 18583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:38.829057 18583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:38.829232 18583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:38.829401 18206 heartbeater.cc:503] Master 127.11.158.190:32879 was elected leader, sending a full tablet report...
I20260926 16:37:38.836549 18583 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.188:45501
I20260926 16:37:38.836841 18734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.188:45501 every 8 connection(s)
I20260926 16:37:38.837576 18583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
I20260926 16:37:38.840561 18735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:38.844465 18735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:37:38.847790 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 18583
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.188@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.158.188@KRBTEST.COM
I20260926 16:37:38.859788 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.846646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.11.158.190:32879)
0926 16:37:38.846797 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.846801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.847434 (+ 633us) server_negotiation.cc:407] Connection header received
0926 16:37:38.848722 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.848727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.848801 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.848892 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.850568 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.851088 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.852426 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.852756 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.852842 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.855687 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.855690 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.855692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.855718 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.857351 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.858063 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.858066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.858068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.858154 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.858554 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.858560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.858564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.858772 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.858886 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.859457 (+ 571us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.859586 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:37:38.860924 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.848767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.11.158.188:45501)
0926 16:37:38.849156 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.849167 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.849188 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:37:38.849224 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.849232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.849422 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.849603 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.850264 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.851408 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.852283 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.852947 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.853122 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.856096 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.856102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.856113 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.856135 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.857976 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.858467 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.858473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.858475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.858560 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.858850 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.858855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.858857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.859215 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.859435 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.859689 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.859933 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":286,"spinlock_wait_cycles":8064,"thread_start_us":165,"threads_started":1}
I20260926 16:37:38.860929 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.846850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:56210)
0926 16:37:38.847295 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.847336 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.848466 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.849100 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.849109 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.849600 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.850424 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.850436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.851237 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.851240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.852274 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.852282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.852896 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.852904 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.853126 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.853922 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.853938 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.855496 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.857465 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.857474 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.857485 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.857941 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.858288 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.858292 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.858294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.858450 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.858885 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.858892 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.859226 (+ 334us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.859403 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.859733 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":159,"spinlock_wait_cycles":6656,"thread_start_us":108,"threads_started":1}
I20260926 16:37:38.862361 18735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.188@KRBTEST.COM for oryx/127.11.158.189@KRBTEST.COM
I20260926 16:37:38.871147 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.862904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.0.0.1:42214)
0926 16:37:38.863004 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.863021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.863099 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.863630 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.863633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.863805 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.863915 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.863920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.864944 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.864947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.865593 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.865602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.866047 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.866051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.866137 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.866586 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.866597 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.867857 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.869720 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.869723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.869725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.869918 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.870268 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.870271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.870273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.870357 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.870736 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.870738 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.870822 (+ 84us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.870905 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.871018 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 16:37:38.871297 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.863106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.11.158.189:46079)
0926 16:37:38.863399 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.863403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.863416 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:38.863451 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.863454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.863493 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.863570 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.864036 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.864821 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.865710 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.865936 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.866013 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.867982 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.867985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.867987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.868009 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.869593 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.870029 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.870032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.870034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.870137 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.870450 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.870452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.870454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.870646 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.870726 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.871001 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.871123 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":123,"threads_started":1}
I20260926 16:37:38.872121 18735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440658, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.188@KRBTEST.COM for oryx/127.11.158.188@KRBTEST.COM
I20260926 16:37:38.880103 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.872682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.0.0.1:35930)
0926 16:37:38.872802 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.872818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.872891 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.873227 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.873230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.873367 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.873450 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.873455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.874315 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.874317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.874925 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.874934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.875384 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.875386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.875443 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.875864 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.875876 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.877196 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.878752 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.878754 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.878756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.878956 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.879249 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.879251 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.879252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.879339 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.879677 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.879679 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.879760 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.879852 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.879956 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 16:37:38.880178 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.872854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.11.158.188:45501)
0926 16:37:38.873009 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.873013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.873026 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:38.873058 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.873061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.873098 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.873173 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.873580 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.874133 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.875055 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.875286 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.875368 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.877322 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.877325 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.877327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.877347 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.878637 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.879075 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.879078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.879079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.879153 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.879420 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.879422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.879423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.879586 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.879661 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.879931 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.880025 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 16:37:38.883700 18735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Bootstrap starting.
I20260926 16:37:38.886402 18735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:38.887302 18735 log.cc:824] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:38.889436 18735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: No bootstrap required, opened a new log
I20260926 16:37:38.892481 18735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } }
I20260926 16:37:38.892694 18735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:38.892787 18735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78cfcea6928e460485d6300e5b396d80, State: Initialized, Role: FOLLOWER
I20260926 16:37:38.893276 18735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } }
I20260926 16:37:38.893792 18741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } } }
I20260926 16:37:38.893939 18741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:37:38.894502 18735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 16:37:38.899354 18752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:37:38.899485 18752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:37:38.902840 18583 master_runner.cc:428] Detected that this master 78cfcea6928e460485d6300e5b396d80 is joining an existing cluster
I20260926 16:37:38.902944 18583 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.158.188:45501
I20260926 16:37:38.914561 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 18077
I20260926 16:37:38.921200 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:43867
--local_ip_for_outbound_sockets=127.11.158.129
--tserver_master_addrs=127.11.158.190:32879
--webserver_port=44591
--webserver_interface=127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab
--principal=oryx/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:37:38.940292 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.906152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.11.158.188:45501)
0926 16:37:38.906248 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.906252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.906903 (+ 651us) server_negotiation.cc:407] Connection header received
0926 16:37:38.907038 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.907040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.907074 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.907117 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.907701 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.908424 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.909875 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.910099 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.910209 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.911538 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.911541 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.911544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.911565 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.938333 (+ 26768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.938850 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.938853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.938855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.938936 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.939259 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.939262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.939264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.939510 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.939581 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.939939 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.940078 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:37:38.941200 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.906326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.0.0.1:35946)
0926 16:37:38.906832 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.906846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.906952 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.907219 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.907221 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.907422 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.907582 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.907591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.908563 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.908566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.909768 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.909777 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.910354 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.910359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.910457 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.910899 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.910913 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.911411 (+ 498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.938454 (+ 27043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.938460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.938465 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.938737 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.939046 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.939050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.939053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.939154 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.939661 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.939663 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.939777 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.940830 (+ 1053us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.940983 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":249,"thread_start_us":76,"threads_started":1}
I20260926 16:37:38.943356 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.905716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.11.158.189:46079)
0926 16:37:38.905855 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.905859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.906110 (+ 251us) server_negotiation.cc:407] Connection header received
0926 16:37:38.906284 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.906290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.906337 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.906389 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.907025 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.907809 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.910564 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.910937 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.911023 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.937815 (+ 26792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.937823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.937827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.937861 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.940897 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.942028 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.942031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.942033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.942114 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.942422 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.942425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.942427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.942645 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.942716 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.943059 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.943189 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:37:38.943657 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.905162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:56216)
0926 16:37:38.905499 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.905512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.905691 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.906141 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.906144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.906387 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.906589 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.906599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.907789 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.907793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.908861 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.908872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.909596 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.909599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.909700 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.910294 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.910312 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.936934 (+ 26622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.941023 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.941026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.941028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.941267 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.941596 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.941599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.941601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.941691 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.943145 (+ 1454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.943149 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.943232 (+ 83us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.943317 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.943451 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":40,"spinlock_wait_cycles":896,"thread_start_us":112,"threads_started":1}
I20260926 16:37:38.944201 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.905252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56216 (local address 127.11.158.190:32879)
0926 16:37:38.905398 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.905405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.905602 (+ 197us) server_negotiation.cc:407] Connection header received
0926 16:37:38.905859 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.905862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.905923 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.906000 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.906741 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.907615 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.909002 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.909360 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.909460 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.937192 (+ 27732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.937200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.937204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.937250 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.939948 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.941399 (+ 1451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.941403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.941405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.941498 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.941801 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.941804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.941806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.942035 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.942137 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.943926 (+ 1789us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.944046 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:37:38.944212 18759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.905609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.0.0.1:42226)
0926 16:37:38.906036 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.906049 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.906176 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.906526 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.906528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.906751 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.906870 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.906881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.907979 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.907983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.909626 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.909635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.911186 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.911190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.911277 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.911689 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.911698 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.937602 (+ 25904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.941693 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.941696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.941697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.941916 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.942224 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.942227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.942228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.942317 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.942788 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.942791 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.942888 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.943913 (+ 1025us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.944046 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":68,"thread_start_us":89,"threads_started":1}
Sep 26 16:37:38 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440657, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.190@KRBTEST.COM for oryx/127.11.158.188@KRBTEST.COM
I20260926 16:37:38.955641 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.947856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.0.0.1:35948)
0926 16:37:38.947995 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.948011 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.948114 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.948392 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.948395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.948572 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.948660 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.948665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.949365 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.949370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.949915 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.949922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.950355 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.950359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.950450 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.950900 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.950921 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.952387 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.954182 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.954187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.954189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.954382 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.954666 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.954671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.954673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.954759 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.955122 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.955125 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.955222 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.955327 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.955473 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 16:37:38.955653 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.947892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.11.158.188:45501)
0926 16:37:38.947993 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.947997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.948060 (+ 63us) server_negotiation.cc:407] Connection header received
0926 16:37:38.948224 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.948227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.948267 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.948387 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.948768 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.949262 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.950027 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.950248 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.950354 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.952542 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.952545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.952548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.952570 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.954067 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.954486 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.954490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.954492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.954571 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.954847 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.954852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.954854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.955022 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.955123 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.955375 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.955471 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:37:38.956981 18015 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.158.188:45501: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "78cfcea6928e460485d6300e5b396d80"
member_type: NON_VOTER
last_known_addr {
host: "127.11.158.188"
port: 45501
}
attrs {
promote: true
}
}
dest_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8"
cas_config_opid_index: 5
I20260926 16:37:38.957542 18015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } }
I20260926 16:37:38.958583 18574 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 78cfcea6928e460485d6300e5b396d80 (127.11.158.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.959610 18709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:38.960192 18574 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.158.188:45501
I20260926 16:37:38.960412 18574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.960501 18574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:38.961122 18709 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Refusing update from remote peer 9b5b1419a34348c4814c9e5f9e9bcbf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 16:37:38.961688 18576 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:38.961807 18583 master.cc:537] Master@127.11.158.188:45501 shutting down...
W20260926 16:37:38.964016 18001 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 -> Peer 78cfcea6928e460485d6300e5b396d80 (127.11.158.188:45501): Couldn't send request to peer 78cfcea6928e460485d6300e5b396d80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 16:37:38.967193 18583 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:38.967486 18583 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:38.967597 18583 tablet_replica.cc:330] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: stopping tablet replica
I20260926 16:37:38.969880 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.958843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.0.0.1:42238)
0926 16:37:38.964384 (+ 5541us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.964401 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.964488 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.965600 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.965604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.965623 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:38.965715 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.965721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.966671 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.966675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.968240 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.968253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.969407 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.969413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.969526 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.969573 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.969659 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.969736 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5430}
I20260926 16:37:38.970355 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.959062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.11.158.189:46079)
0926 16:37:38.961611 (+ 2549us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.961616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.964571 (+ 2955us) server_negotiation.cc:407] Connection header received
0926 16:37:38.964630 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.964633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.964694 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.964773 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:38.965852 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.966522 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.968432 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.969038 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.969140 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.969302 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.970134 (+ 832us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.970223 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2455}
I20260926 16:37:38.971400 18627 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 LEARNER]: Refusing update from remote peer 9b5b1419a34348c4814c9e5f9e9bcbf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 16:37:38.971812 18576 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 16:37:38.973038 18583 master.cc:552] Master@127.11.158.188:45501 shutdown complete.
I20260926 16:37:38.973085 18583 master_runner.cc:305] Clearing existing system tablet
I20260926 16:37:38.973322 18583 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:38.973372 18583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:38.973764 18627 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 78cfcea6928e460485d6300e5b396d80 (127.11.158.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.974637 18583 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 16:37:38.975448 18764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:38.975621 18583 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:38.975745 18583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:38.975777 18583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
uuid: "78cfcea6928e460485d6300e5b396d80"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "a207fb6e213603e3f2c3e8bc7fb26dd5"
server_key_iv: "babc29a2d45d1a1e58553118b7bc24ab"
server_key_version: "encryptionkey@0"
I20260926 16:37:38.975986 18656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b5b1419a34348c4814c9e5f9e9bcbf8. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.976152 18656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:37:38.976243 18656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: NON_VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: true } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.976313 18656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:37:38.977648 18583 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:38.979467 18583 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 16:37:38.979576 18583 log.cc:1196] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 16:37:38.979815 18583 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Deleting consensus metadata
I20260926 16:37:38.979995 18583 master_runner.cc:315] Copying system tablet from 127.11.158.190:32879
I20260926 16:37:38.981168 18583 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: tablet copy: Beginning tablet copy session from remote peer at address 127.11.158.190:32879
I20260926 16:37:38.986683 18576 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: attempting to promote NON_VOTER 0e8c28627ef94c9d824564d7b1fba272 to VOTER
I20260926 16:37:38.987275 18576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } }
I20260926 16:37:38.989343 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.988943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.0.0.1:35952)
0926 16:37:38.989175 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.989251 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260926 16:37:38.989717 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.981736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:56232)
0926 16:37:38.982015 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.982029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:38.982167 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:38.982404 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:38.982407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:38.982621 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:38.982716 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.982721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.983482 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.983488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.984062 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:38.984071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.984634 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.984638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:38.984743 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.985296 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:38.985308 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:38.985631 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:38.987658 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.987661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.987664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.987907 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.988247 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:38.988250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:38.988252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:38.988418 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:38.988902 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:38.988904 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:38.989008 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 16:37:38.989377 (+ 369us) client_negotiation.cc:240] Negotiation successful
0926 16:37:38.989533 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":68,"threads_started":1}
I20260926 16:37:38.989912 18627 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 LEARNER]: Refusing update from remote peer 9b5b1419a34348c4814c9e5f9e9bcbf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 16:37:38.990805 18575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 16:37:38.990890 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.990674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.0.0.1:35968)
0926 16:37:38.990782 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:38.990836 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 16:37:38.990988 18001 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 -> Peer 78cfcea6928e460485d6300e5b396d80 (127.11.158.188:45501): Couldn't send request to peer 78cfcea6928e460485d6300e5b396d80. Status: Network error: Client connection negotiation failed: client connection to 127.11.158.188:45501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 16:37:38.991261 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:38.981808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.11.158.190:32879)
0926 16:37:38.981973 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:38.981978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:38.982125 (+ 147us) server_negotiation.cc:407] Connection header received
0926 16:37:38.982247 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:38.982250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:38.982296 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:38.982389 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:38.982834 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.983372 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.984187 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:38.984503 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:38.984610 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:38.985771 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:38.985775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:38.985778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:38.985804 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:38.987534 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.988022 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.988025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.988027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.988126 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:38.988525 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:38.988529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:38.988531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:38.988750 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:38.988825 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:38.989200 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 16:37:38.989311 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 16:37:38.991703 18035 tablet_copy_service.cc:140] P 9b5b1419a34348c4814c9e5f9e9bcbf8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 78cfcea6928e460485d6300e5b396d80 ({username='oryx', principal='oryx/127.11.158.188@KRBTEST.COM'} at 127.0.0.1:56232)
I20260926 16:37:38.991787 18035 tablet_copy_service.cc:161] P 9b5b1419a34348c4814c9e5f9e9bcbf8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 78cfcea6928e460485d6300e5b396d80 at {username='oryx', principal='oryx/127.11.158.188@KRBTEST.COM'} at 127.0.0.1:56232: session id = 78cfcea6928e460485d6300e5b396d80-00000000000000000000000000000000
I20260926 16:37:38.992602 18575 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0e8c28627ef94c9d824564d7b1fba272 (127.11.158.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.993927 18575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.994035 18575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:38.994174 18575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.994262 18575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:38.994853 18627 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0e8c28627ef94c9d824564d7b1fba272 (127.11.158.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.996950 18035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 16:37:38.997354 18765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:38.997473 18765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:37:38.998230 18583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 16:37:39.000099 18583 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: tablet copy: Starting download of 0 data blocks...
I20260926 16:37:39.000313 18583 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: tablet copy: Starting download of 1 WAL segments...
I20260926 16:37:39.001874 18583 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 16:37:39.003752 18035 tablet_copy_service.cc:342] P 9b5b1419a34348c4814c9e5f9e9bcbf8: Request end of tablet copy session 78cfcea6928e460485d6300e5b396d80-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.158.188@KRBTEST.COM'} at 127.0.0.1:56232
I20260926 16:37:39.003893 18035 tablet_copy_service.cc:433] P 9b5b1419a34348c4814c9e5f9e9bcbf8: ending tablet copy session 78cfcea6928e460485d6300e5b396d80-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 78cfcea6928e460485d6300e5b396d80
I20260926 16:37:39.011471 18583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:39.013842 18775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:39.013842 18772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:39.014245 18583 server_base.cc:1034] running on GCE node
W20260926 16:37:39.017088 18773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:39.017426 18583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:39.017884 18583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:39.019055 18583 hybrid_clock.cc:648] HybridClock initialized: now 1790440659019040 us; error 44 us; skew 500 ppm
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:39.020675 18583 init.cc:377] Logged in from keytab as oryx/127.11.158.188@KRBTEST.COM (short username oryx)
I20260926 16:37:39.021261 18583 webserver.cc:460] Webserver started at http://127.11.158.188:42047/ using document root <none> and password file <none>
I20260926 16:37:39.021466 18583 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:39.021521 18583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:39.022769 18583 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:39.023742 18782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.023968 18583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:39.024042 18583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
uuid: "78cfcea6928e460485d6300e5b396d80"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "a207fb6e213603e3f2c3e8bc7fb26dd5"
server_key_iv: "babc29a2d45d1a1e58553118b7bc24ab"
server_key_version: "encryptionkey@0"
I20260926 16:37:39.024153 18583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:39.034340 18583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:39.036734 18583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:39.041708 18583 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.188:45501
I20260926 16:37:39.041721 18834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.188:45501 every 8 connection(s)
I20260926 16:37:39.042290 18583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
I20260926 16:37:39.044088 18835 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:37:39.045123 18835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Bootstrap starting.
I20260926 16:37:39.050032 18340 heartbeater.cc:503] Master 127.11.158.190:32879 was elected leader, sending a full tablet report...
I20260926 16:37:39.055104 18835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:37:39.055408 18835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: Bootstrap complete.
I20260926 16:37:39.055918 18835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } }
I20260926 16:37:39.056056 18835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 78cfcea6928e460485d6300e5b396d80, State: Initialized, Role: LEARNER
I20260926 16:37:39.056174 18835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } }
I20260926 16:37:39.056517 18838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:39.056679 18838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:37:39.056689 18835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:39.060655 18849 catalog_manager.cc:1259] Loaded cluster ID: 0d6648a44d70470890f55acda11ed1ab
I20260926 16:37:39.060720 18849 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: loading cluster ID for follower catalog manager: success
I20260926 16:37:39.062282 18849 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: acquiring CA information for follower catalog manager: success
I20260926 16:37:39.063352 18849 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 16:37:39.087155 18761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:39.087498 18761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:39.087560 18761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:39.093794 18761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:39.093881 18761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:39.093976 18761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:39.102653 18761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab
--principal=oryx/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:43867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=44591
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:39.104200 18761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:39.105691 18761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:39.116272 18855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:39.116460 18761 server_base.cc:1034] running on GCE node
W20260926 16:37:39.116256 18857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:39.116365 18854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:39.117071 18761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:39.117669 18761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:39.118827 18761 hybrid_clock.cc:648] HybridClock initialized: now 1790440659118794 us; error 46 us; skew 500 ppm
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:39.121628 18761 init.cc:377] Logged in from keytab as oryx/127.11.158.129@KRBTEST.COM (short username oryx)
I20260926 16:37:39.122479 18761 webserver.cc:460] Webserver started at http://127.11.158.129:44591/ using document root <none> and password file <none>
I20260926 16:37:39.123042 18761 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:39.123144 18761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:39.126964 18761 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:39.129117 18864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.130429 18761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:39.130566 18761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "676b8797be7847339c9336fe0e2b191f"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "374e8cecb7adb6d29af2bbeefa5f0716"
server_key_iv: "fe1e903185df6a0a5c54b8a661dc65a4"
server_key_version: "encryptionkey@0"
I20260926 16:37:39.131147 18761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:39.152772 18761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:39.156003 18761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:39.156157 18761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:39.156821 18761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:39.157694 18761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:39.157735 18761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.157774 18761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:39.157820 18761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.166796 18761 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:43867
I20260926 16:37:39.166849 18977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:43867 every 8 connection(s)
I20260926 16:37:39.167824 18761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:39.176463 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 18761
I20260926 16:37:39.176692 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 18211
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.189@KRBTEST.COM
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.188@KRBTEST.COM
I20260926 16:37:39.186319 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:46573
--local_ip_for_outbound_sockets=127.11.158.130
--tserver_master_addrs=127.11.158.190:32879
--webserver_port=46493
--webserver_interface=127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab
--principal=oryx/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:37:39.186553 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.169755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:54243 (local address 127.11.158.188:45501)
0926 16:37:39.170285 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.170291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.171489 (+ 1198us) server_negotiation.cc:407] Connection header received
0926 16:37:39.173193 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.173198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.173255 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.173321 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.174654 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.175710 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.176972 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.177439 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.177558 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.182095 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.182099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.182104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.182138 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.183980 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184693 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.184697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.184698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.184802 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.185147 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.185150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.185152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.185371 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.185452 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.186242 (+ 790us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.186375 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":91,"threads_started":1}
I20260926 16:37:39.186548 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.171985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:33801 (local address 127.11.158.190:32879)
0926 16:37:39.172170 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.172174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.172265 (+ 91us) server_negotiation.cc:407] Connection header received
0926 16:37:39.173194 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.173198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.173255 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.173318 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.174654 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.175489 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.176945 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.177311 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.177393 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.181134 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.181142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.181148 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.181192 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.183980 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184689 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.184694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.184696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.184802 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.185156 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.185160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.185161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.185369 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.185452 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.186244 (+ 792us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.186374 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 16:37:39.186640 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.172405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:35011 (local address 127.11.158.189:46079)
0926 16:37:39.172543 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.172549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.172765 (+ 216us) server_negotiation.cc:407] Connection header received
0926 16:37:39.173155 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.173158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.173211 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.173289 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.174632 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.175153 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.177535 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.177854 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.177938 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.181668 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.181676 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.181682 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.181721 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.183670 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184491 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.184498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.184503 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.184604 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.185073 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.185080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.185085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.185362 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.185470 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.186272 (+ 802us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.186401 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:37:39.188326 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.172394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.11.158.129:35011)
0926 16:37:39.172632 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.172653 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.172951 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.173458 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.173466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.173932 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.174477 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.174491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.175292 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.175295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.177406 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.177414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.178139 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.178144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.178249 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.178854 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.179033 (+ 179us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.181510 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.183834 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.183844 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.183865 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.184299 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.184731 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.184742 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.184941 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.185537 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.185543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.185939 (+ 396us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.186828 (+ 889us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.187297 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":928,"spinlock_wait_cycles":28672,"thread_start_us":96,"threads_started":1}
I20260926 16:37:39.188586 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.171853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.129:33801)
0926 16:37:39.172147 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.172164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.172940 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.173481 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.173484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.173938 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.174480 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.174495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.175605 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.175608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.176806 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.176820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.177599 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.177602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.177822 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.178721 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.178750 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.180877 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.184112 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.184118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.184566 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.184912 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.184917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.185037 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.185524 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.185530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.185938 (+ 408us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.188166 (+ 2228us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.188304 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":248,"spinlock_wait_cycles":66432,"thread_start_us":88,"threads_started":1}
I20260926 16:37:39.189087 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.170369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.11.158.129:54243)
0926 16:37:39.171323 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.171370 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.172940 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.173481 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.173484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.173937 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.174480 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.174496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.175834 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.175837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.176813 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.176829 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.177750 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.177757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.177923 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.179038 (+ 1115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.179062 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.181914 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.184112 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.184118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.184566 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.184912 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.184915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.184917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.185047 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.185524 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.185530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.185938 (+ 408us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.186716 (+ 778us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.187190 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":181,"spinlock_wait_cycles":65024,"thread_start_us":569,"threads_started":1}
I20260926 16:37:39.189893 18979 heartbeater.cc:347] Connected to a master server at 127.11.158.190:32879
I20260926 16:37:39.189996 18980 heartbeater.cc:347] Connected to a master server at 127.11.158.189:46079
I20260926 16:37:39.190018 18978 heartbeater.cc:347] Connected to a master server at 127.11.158.188:45501
I20260926 16:37:39.190146 18980 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.190145 18978 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.190145 18979 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.190779 18980 heartbeater.cc:511] Master 127.11.158.189:46079 requested a full tablet report, sending...
I20260926 16:37:39.190780 18979 heartbeater.cc:511] Master 127.11.158.190:32879 requested a full tablet report, sending...
I20260926 16:37:39.190778 18978 heartbeater.cc:511] Master 127.11.158.188:45501 requested a full tablet report, sending...
I20260926 16:37:39.192236 18015 ts_manager.cc:194] Re-registered known tserver with Master: 676b8797be7847339c9336fe0e2b191f (127.11.158.129:43867)
I20260926 16:37:39.192425 18799 ts_manager.cc:194] Registered new tserver with Master: 676b8797be7847339c9336fe0e2b191f (127.11.158.129:43867)
I20260926 16:37:39.193023 18015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:33801
I20260926 16:37:39.193933 18613 ts_manager.cc:194] Registered new tserver with Master: 676b8797be7847339c9336fe0e2b191f (127.11.158.129:43867)
I20260926 16:37:39.267503 18474 heartbeater.cc:503] Master 127.11.158.190:32879 was elected leader, sending a full tablet report...
W20260926 16:37:39.303305 18986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:39.303608 18986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:39.303656 18986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:39.308053 18986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:39.308125 18986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:39.308224 18986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:39.314519 18986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab
--principal=oryx/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:46573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=46493
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:39.315922 18986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:39.317088 18986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:39.327059 18992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:39.327080 18991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:39.327059 18994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:39.327266 18986 server_base.cc:1034] running on GCE node
I20260926 16:37:39.327841 18986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:39.328565 18986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:39.329751 18986 hybrid_clock.cc:648] HybridClock initialized: now 1790440659329712 us; error 50 us; skew 500 ppm
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:39.332635 18986 init.cc:377] Logged in from keytab as oryx/127.11.158.130@KRBTEST.COM (short username oryx)
I20260926 16:37:39.333535 18986 webserver.cc:460] Webserver started at http://127.11.158.130:46493/ using document root <none> and password file <none>
I20260926 16:37:39.334146 18986 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:39.334224 18986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:39.338106 18986 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 16:37:39.340070 19001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.341358 18986 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:39.341461 18986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "5d5331360dd240e09eb6b57453ede29b"
format_stamp: "Formatted at 2026-09-26 16:37:37 on dist-test-slave-p39x"
server_key: "17e812281e1b2806a4398812e8ebdf2e"
server_key_iv: "7b2f7d6eafb3a36bbf9abf63589b5ded"
server_key_version: "encryptionkey@0"
I20260926 16:37:39.341975 18986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:39.353171 18986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:39.356109 18986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:39.356405 18986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:39.356972 18986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:39.357950 18986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:39.358016 18986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.358078 18986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:39.358129 18986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.366971 18986 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:46573
I20260926 16:37:39.367141 19114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:46573 every 8 connection(s)
I20260926 16:37:39.368008 18986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:39.375204 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 18986
I20260926 16:37:39.375378 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 18345
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.188@KRBTEST.COM
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.189@KRBTEST.COM
I20260926 16:37:39.384626 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:40297
--local_ip_for_outbound_sockets=127.11.158.131
--tserver_master_addrs=127.11.158.190:32879
--webserver_port=38769
--webserver_interface=127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab
--principal=oryx/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:37:39.385026 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.372107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:34741 (local address 127.11.158.190:32879)
0926 16:37:39.372249 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.372254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.372395 (+ 141us) server_negotiation.cc:407] Connection header received
0926 16:37:39.372595 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.372605 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.372660 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.372751 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.374871 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.375638 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.376800 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.377114 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.377181 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.380329 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.380353 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.380359 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.380391 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.382584 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.383178 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.383182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.383184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.383271 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.383686 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.383692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.383695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.383915 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.383977 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.384693 (+ 716us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.384820 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:37:39.385355 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.369833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:58639 (local address 127.11.158.189:46079)
0926 16:37:39.369973 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.369977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.371298 (+ 1321us) server_negotiation.cc:407] Connection header received
0926 16:37:39.372506 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.372509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.372568 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.372623 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.374196 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.374721 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.376822 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.377252 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.377344 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.381608 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.381612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.381614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.381644 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.383508 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.383973 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.383976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.383978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.384056 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.384384 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.384388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.384389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.384633 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.384711 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.385027 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.385156 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:37:39.385695 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.371735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:54775 (local address 127.11.158.188:45501)
0926 16:37:39.371904 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.371909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.372033 (+ 124us) server_negotiation.cc:407] Connection header received
0926 16:37:39.372459 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.372465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.372523 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.372600 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.374209 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.374742 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.376566 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.376925 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.376994 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.381576 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.381582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.381588 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.381621 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.383691 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.384166 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.384169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.384171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.384246 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.384595 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.384599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.384600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.384848 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.384935 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.385295 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.385438 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:37:39.387152 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.371593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.11.158.130:54775)
0926 16:37:39.371937 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.371951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.372229 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.372894 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.372903 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.373460 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.374064 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.374070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.374878 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.374881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.376435 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.376445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.377198 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.377202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.377438 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.378190 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.378213 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.381418 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.383813 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.383818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.383822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.384061 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.384398 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.384400 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.384402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.384487 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.385009 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.385014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.385115 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.385907 (+ 792us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.386086 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":211,"spinlock_wait_cycles":72704,"thread_start_us":77,"threads_started":1}
I20260926 16:37:39.387708 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.371958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:34741)
0926 16:37:39.372289 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.372302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.372475 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.372901 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.372904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.373456 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.374048 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.374061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.375769 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.375772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.376687 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.376695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.377317 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.377320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.377442 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.378464 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.378479 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.380173 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.382698 (+ 2525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.382704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.382718 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.383058 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.383387 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.383390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.383392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.383551 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.384051 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.384056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.384388 (+ 332us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.385888 (+ 1500us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.385994 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":205,"spinlock_wait_cycles":5376,"thread_start_us":82,"threads_started":1}
I20260926 16:37:39.388450 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.370244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.11.158.130:58639)
0926 16:37:39.371131 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.371172 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.372223 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.372894 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.372903 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.373460 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.374061 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.374070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.374864 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.374870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.376680 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.376693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.377531 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.377537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.377674 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.378324 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.378344 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.381428 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.383645 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.383648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.383652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.383870 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.384167 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.384169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.384171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.384242 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.384763 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.384766 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.384850 (+ 84us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.385563 (+ 713us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.385925 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":371,"spinlock_wait_cycles":24960,"thread_start_us":528,"threads_started":1}
I20260926 16:37:39.388490 19115 heartbeater.cc:347] Connected to a master server at 127.11.158.188:45501
I20260926 16:37:39.388724 19115 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.388988 19116 heartbeater.cc:347] Connected to a master server at 127.11.158.190:32879
I20260926 16:37:39.389062 19116 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.389235 19116 heartbeater.cc:511] Master 127.11.158.190:32879 requested a full tablet report, sending...
I20260926 16:37:39.389233 19115 heartbeater.cc:511] Master 127.11.158.188:45501 requested a full tablet report, sending...
I20260926 16:37:39.389748 19117 heartbeater.cc:347] Connected to a master server at 127.11.158.189:46079
I20260926 16:37:39.389814 19117 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.389959 19117 heartbeater.cc:511] Master 127.11.158.189:46079 requested a full tablet report, sending...
I20260926 16:37:39.390339 18015 ts_manager.cc:194] Re-registered known tserver with Master: 5d5331360dd240e09eb6b57453ede29b (127.11.158.130:46573)
I20260926 16:37:39.390455 18799 ts_manager.cc:194] Registered new tserver with Master: 5d5331360dd240e09eb6b57453ede29b (127.11.158.130:46573)
I20260926 16:37:39.390931 18015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:34741
I20260926 16:37:39.391789 18613 ts_manager.cc:194] Registered new tserver with Master: 5d5331360dd240e09eb6b57453ede29b (127.11.158.130:46573)
I20260926 16:37:39.393679 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.386712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.11.158.188:45501)
0926 16:37:39.386830 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.386834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.386847 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:39.386939 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.386942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.387008 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.387126 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:39.387533 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.388265 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.392779 (+ 4514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.393194 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.393323 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.393455 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.393497 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.393561 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:37:39.393682 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.386622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.0.0.1:35976)
0926 16:37:39.386774 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.386787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.386858 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.387263 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.387268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.387292 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:39.387394 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.387400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.388407 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.388410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.389497 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.389507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.393316 (+ 3809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.393320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.393403 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.393424 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.393488 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.393565 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260926 16:37:39.394217 18809 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 16:37:39.394717 18838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b5b1419a34348c4814c9e5f9e9bcbf8. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:39.394842 18838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: This master's current role is: LEARNER
W20260926 16:37:39.497896 19122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:39.498176 19122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:39.498257 19122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:39.503247 19122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:39.503363 19122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:39.503458 19122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:39.509693 19122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab
--principal=oryx/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:40297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=38769
--tserver_master_addrs=127.11.158.190:32879,127.11.158.189:46079,127.11.158.188:45501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:39.510819 19122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:39.511861 19122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:39.522220 19130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:39.522220 19127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:39.522325 19122 server_base.cc:1034] running on GCE node
W20260926 16:37:39.522259 19128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:39.522998 19122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:39.523685 19122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:39.524859 19122 hybrid_clock.cc:648] HybridClock initialized: now 1790440659524816 us; error 67 us; skew 500 ppm
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:39.527617 19122 init.cc:377] Logged in from keytab as oryx/127.11.158.131@KRBTEST.COM (short username oryx)
I20260926 16:37:39.528581 19122 webserver.cc:460] Webserver started at http://127.11.158.131:38769/ using document root <none> and password file <none>
I20260926 16:37:39.529104 19122 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:39.529174 19122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:39.532929 19122 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260926 16:37:39.535123 19137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.536713 19122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:39.536824 19122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "f1bf0a9157a74f05a449cd80a6bf4af4"
format_stamp: "Formatted at 2026-09-26 16:37:38 on dist-test-slave-p39x"
server_key: "124622dc045f85eb45f687fd29bc192b"
server_key_iv: "ec6a5531e68cc7e9d8e0fab8988d8474"
server_key_version: "encryptionkey@0"
I20260926 16:37:39.537493 19122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:39.569065 19122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:39.572356 19122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:39.572503 19122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:39.573022 19122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:39.573931 19122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:39.573978 19122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.574018 19122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:39.574033 19122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:39.582966 19122 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:40297
I20260926 16:37:39.582989 19250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:40297 every 8 connection(s)
I20260926 16:37:39.583979 19122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:39.594429 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 19122
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.188@KRBTEST.COM
Sep 26 16:37:39 dist-test-slave-p39x krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440659, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.189@KRBTEST.COM
I20260926 16:37:39.602963 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.586227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.11.158.131:52259)
0926 16:37:39.587178 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.587224 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.588281 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.588871 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.588881 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.589523 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.590131 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.590140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.591211 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.591218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.592156 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.592165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.592911 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.592915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.593251 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.594201 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.594214 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.596287 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.599316 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.599323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.599343 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.599712 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.600001 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.600004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.600006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.600177 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.600676 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.600682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.601016 (+ 334us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.601370 (+ 354us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.601760 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":307,"spinlock_wait_cycles":25216,"thread_start_us":581,"threads_started":1}
I20260926 16:37:39.603590 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.585752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:52259 (local address 127.11.158.188:45501)
0926 16:37:39.585956 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.585960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.587329 (+ 1369us) server_negotiation.cc:407] Connection header received
0926 16:37:39.588533 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.588543 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.588605 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.588676 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.590289 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.591097 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.592305 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.592669 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.592769 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.596506 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.596510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.596513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.596549 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.599181 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.599822 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.599826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.599828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.599909 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.600278 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.600281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.600282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.600521 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.600624 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.603279 (+ 2655us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.603424 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260926 16:37:39.603165 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.588015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.11.158.131:33773)
0926 16:37:39.588695 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.588707 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.588807 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.589165 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.589167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.589529 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.590132 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.590140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.591665 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.591669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.592517 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.592524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.593020 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.593025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.593260 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.594099 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.594111 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.596924 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.600110 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.600113 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.600115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.600372 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.600680 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.600683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.600685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.600766 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.601415 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.601418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.601507 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.601626 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.601990 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":122,"spinlock_wait_cycles":18048,"thread_start_us":103,"threads_started":1}
I20260926 16:37:39.605311 19251 heartbeater.cc:347] Connected to a master server at 127.11.158.188:45501
I20260926 16:37:39.605422 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.588315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:33773 (local address 127.11.158.189:46079)
0926 16:37:39.588468 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.588473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.588800 (+ 327us) server_negotiation.cc:407] Connection header received
0926 16:37:39.588881 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.588884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.588937 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.589004 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.590289 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.591094 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.592635 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.592910 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.593016 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.597093 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.597099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.597102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.597135 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.599990 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.600481 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.600486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.600489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.600569 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.600867 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.600870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.600872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.601268 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.601371 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.605107 (+ 3736us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.605237 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 16:37:39.605568 19251 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.606094 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.587671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.131:53361)
0926 16:37:39.588012 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.588025 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.588281 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.588974 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.588977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.589494 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:39.590119 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.590135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.590970 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.590975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.592263 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.592272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.592910 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.592914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.593200 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.593874 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:39.593899 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:39.596023 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:39.603930 (+ 7907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.603933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.603935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.604489 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.604829 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:39.604832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:39.604834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:39.604920 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:39.605627 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:39.605629 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:39.605720 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.605810 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.605946 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":139,"thread_start_us":76,"threads_started":1}
I20260926 16:37:39.606146 19251 heartbeater.cc:511] Master 127.11.158.188:45501 requested a full tablet report, sending...
I20260926 16:37:39.606758 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.587806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:53361 (local address 127.11.158.190:32879)
0926 16:37:39.587975 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.587980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.588124 (+ 144us) server_negotiation.cc:407] Connection header received
0926 16:37:39.588813 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.588819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.588873 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.588950 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.590279 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.590825 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.592414 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.592765 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.592868 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.596244 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.596248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.596251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.596286 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.603792 (+ 7506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.604580 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.604583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.604611 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.604718 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.605127 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.605130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.605132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.605380 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.605469 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.606403 (+ 934us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.606571 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 16:37:39.607309 19253 heartbeater.cc:347] Connected to a master server at 127.11.158.189:46079
I20260926 16:37:39.607393 19253 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.607563 19253 heartbeater.cc:511] Master 127.11.158.189:46079 requested a full tablet report, sending...
I20260926 16:37:39.607605 18799 ts_manager.cc:194] Registered new tserver with Master: f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297)
I20260926 16:37:39.608685 18613 ts_manager.cc:194] Registered new tserver with Master: f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297)
I20260926 16:37:39.609128 19252 heartbeater.cc:347] Connected to a master server at 127.11.158.190:32879
I20260926 16:37:39.609198 19252 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:39.609359 19266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.597970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35984 (local address 127.11.158.188:45501)
0926 16:37:39.598193 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.598200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.598448 (+ 248us) server_negotiation.cc:407] Connection header received
0926 16:37:39.598564 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.598567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.598610 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.598708 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.599452 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.600233 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.601531 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.601865 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.601941 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.603774 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.603779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.603784 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.603812 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.606676 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.607193 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.607197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.607198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.607285 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.608380 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.608384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.608385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.608593 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.608700 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.609071 (+ 371us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.609204 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260926 16:37:39.609365 19252 heartbeater.cc:511] Master 127.11.158.190:32879 requested a full tablet report, sending...
I20260926 16:37:39.609946 18015 ts_manager.cc:194] Re-registered known tserver with Master: f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297)
I20260926 16:37:39.610702 18015 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:53361
I20260926 16:37:39.612277 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.597618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.11.158.189:46079)
0926 16:37:39.598255 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.598262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.598278 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:39.598325 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.598328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.598378 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.598471 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.599556 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.600385 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.601875 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.602260 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.602399 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.604638 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.604642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.604644 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.604674 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.607035 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.609958 (+ 2923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.609964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.609968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.610063 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.610361 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.610365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.610367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.610591 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.610710 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.610997 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.612113 (+ 1116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":60,"threads_started":1}
I20260926 16:37:39.615103 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.596797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.11.158.190:32879)
0926 16:37:39.607133 (+ 10336us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.607137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.607151 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:37:39.607188 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.607191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.607235 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.607870 (+ 635us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.608167 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.608940 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.610095 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.610418 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.610522 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.612011 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.612014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.612016 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.612036 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.613740 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.614154 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.614156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.614157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.614222 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.614445 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.614447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.614448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.614589 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.614688 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.614848 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.614926 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10260,"thread_start_us":97,"threads_started":1}
I20260926 16:37:39.617627 18015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:39.620568 18015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:39.622887 18576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: NON_VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: true } } }
I20260926 16:37:39.623028 18576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:39.626514 19269 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: attempting to promote NON_VOTER 78cfcea6928e460485d6300e5b396d80 to VOTER
I20260926 16:37:39.626860 19269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } }
I20260926 16:37:39.627979 18809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 LEARNER]: Refusing update from remote peer 9b5b1419a34348c4814c9e5f9e9bcbf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 16:37:39.628006 18627 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Refusing update from remote peer 9b5b1419a34348c4814c9e5f9e9bcbf8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 16:37:39.628417 19270 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260926 16:37:39.628705 19270 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260926 16:37:39.630267 18576 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 78cfcea6928e460485d6300e5b396d80 (127.11.158.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.630517 18627 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 78cfcea6928e460485d6300e5b396d80 (127.11.158.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.631820 18576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.631925 18576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:39.632084 18576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.632112 19270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.631810 18809 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 78cfcea6928e460485d6300e5b396d80 (127.11.158.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.632184 19270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:39.632174 18576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:39.633769 18838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.633922 18838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:37:39.634159 19279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:39.634305 19279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:37:39.642340 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.636584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.11.158.130:46573)
0926 16:37:39.636914 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.636924 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.637668 (+ 744us) server_negotiation.cc:407] Connection header received
0926 16:37:39.637884 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.637890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.638080 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.638252 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:39.638417 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.639462 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.641143 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.641788 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.641924 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.642063 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.642124 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.642212 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":118,"threads_started":1}
I20260926 16:37:39.642300 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.636216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:46573 (local address 127.0.0.1:32896)
0926 16:37:39.637579 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.637592 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.637738 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.638211 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.638214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.638229 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:39.638317 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.638322 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.639573 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.639577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.640995 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.641004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.641909 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.641913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.641993 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.642016 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.642080 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.642151 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1307,"mutex_wait_us":63,"thread_start_us":74,"threads_started":1}
I20260926 16:37:39.642670 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.636176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43867 (local address 127.0.0.1:52602)
0926 16:37:39.637258 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.637270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.637387 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.637896 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.637899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.637917 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:39.638044 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.638051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.639454 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.639460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.640899 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.640908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.642006 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.642011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.642410 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.642431 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.642497 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.642571 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":582,"thread_start_us":108,"threads_started":1}
I20260926 16:37:39.642493 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.635998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:40297 (local address 127.0.0.1:54246)
0926 16:37:39.636156 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.636173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.636258 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.638449 (+ 2191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.638452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.638467 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:39.638550 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.638555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.639486 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.639488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.640993 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.641004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.642015 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.642018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.642130 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.642152 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.642216 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.642284 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":246}
I20260926 16:37:39.643435 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.636141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.11.158.131:40297)
0926 16:37:39.638007 (+ 1866us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.638020 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.638040 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:37:39.638116 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.638124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.638299 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.638487 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:39.638632 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.639339 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.641096 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.641879 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.642971 (+ 1092us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.643111 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.643177 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.643280 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1772,"thread_start_us":325,"threads_started":1}
I20260926 16:37:39.644650 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.636906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.11.158.129:43867)
0926 16:37:39.637140 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.637172 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.637386 (+ 214us) server_negotiation.cc:407] Connection header received
0926 16:37:39.637488 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.637493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.637738 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.637914 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:39.638167 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.639327 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.641022 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.641860 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.642126 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.642307 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.644223 (+ 1916us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.644325 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":56,"threads_started":1}
I20260926 16:37:39.645720 19049 tablet_service.cc:1467] Processing CreateTablet for tablet 10fc1ee39e1043deba4a636d4355d79b (DEFAULT_TABLE table=test-table [id=52f9bd00b05a477b955f5f74aabd42af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:39.646363 19185 tablet_service.cc:1467] Processing CreateTablet for tablet 10fc1ee39e1043deba4a636d4355d79b (DEFAULT_TABLE table=test-table [id=52f9bd00b05a477b955f5f74aabd42af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:39.646858 19049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fc1ee39e1043deba4a636d4355d79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:39.647267 19185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fc1ee39e1043deba4a636d4355d79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:39.647344 18912 tablet_service.cc:1467] Processing CreateTablet for tablet 10fc1ee39e1043deba4a636d4355d79b (DEFAULT_TABLE table=test-table [id=52f9bd00b05a477b955f5f74aabd42af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:39.648284 18912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fc1ee39e1043deba4a636d4355d79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:39.654793 19285 tablet_bootstrap.cc:492] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Bootstrap starting.
I20260926 16:37:39.655575 19286 tablet_bootstrap.cc:492] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Bootstrap starting.
I20260926 16:37:39.656653 19287 tablet_bootstrap.cc:492] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Bootstrap starting.
I20260926 16:37:39.657135 19285 tablet_bootstrap.cc:654] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:39.657843 19286 tablet_bootstrap.cc:654] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:39.658010 19285 log.cc:824] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:39.658744 19287 tablet_bootstrap.cc:654] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:39.658695 19286 log.cc:824] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:39.659566 19287 log.cc:824] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:39.660521 19285 tablet_bootstrap.cc:492] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: No bootstrap required, opened a new log
I20260926 16:37:39.660770 19285 ts_tablet_manager.cc:1397] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:37:39.661315 19286 tablet_bootstrap.cc:492] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: No bootstrap required, opened a new log
I20260926 16:37:39.661350 19287 tablet_bootstrap.cc:492] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: No bootstrap required, opened a new log
I20260926 16:37:39.661499 19286 ts_tablet_manager.cc:1397] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:39.661535 19287 ts_tablet_manager.cc:1397] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:37:39.664806 19287 raft_consensus.cc:348] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.665057 19287 raft_consensus.cc:374] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:39.664866 19285 raft_consensus.cc:348] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.665118 19287 raft_consensus.cc:729] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676b8797be7847339c9336fe0e2b191f, State: Initialized, Role: FOLLOWER
I20260926 16:37:39.665165 19285 raft_consensus.cc:374] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:39.665220 19285 raft_consensus.cc:729] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bf0a9157a74f05a449cd80a6bf4af4, State: Initialized, Role: FOLLOWER
I20260926 16:37:39.665571 19287 consensus_queue.cc:260] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.665685 19285 consensus_queue.cc:260] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.665925 19286 raft_consensus.cc:348] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.666167 19286 raft_consensus.cc:374] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:39.666218 19286 raft_consensus.cc:729] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5331360dd240e09eb6b57453ede29b, State: Initialized, Role: FOLLOWER
I20260926 16:37:39.666615 19287 ts_tablet_manager.cc:1428] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260926 16:37:39.666625 19285 ts_tablet_manager.cc:1428] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:37:39.666635 19286 consensus_queue.cc:260] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.666973 19252 heartbeater.cc:503] Master 127.11.158.190:32879 was elected leader, sending a full tablet report...
I20260926 16:37:39.667855 18979 heartbeater.cc:503] Master 127.11.158.190:32879 was elected leader, sending a full tablet report...
W20260926 16:37:39.670310 18981 tablet.cc:2369] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:39.671447 19116 heartbeater.cc:503] Master 127.11.158.190:32879 was elected leader, sending a full tablet report...
I20260926 16:37:39.671850 19286 ts_tablet_manager.cc:1428] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260926 16:37:39.835196 19254 tablet.cc:2369] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:37:39.870090 19118 tablet.cc:2369] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:39.885396 19291 raft_consensus.cc:482] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:39.885701 19291 raft_consensus.cc:504] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.887132 19291 leader_election.cc:288] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d5331360dd240e09eb6b57453ede29b (127.11.158.130:46573), f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297)
I20260926 16:37:39.892835 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.887614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:43609 (local address 127.11.158.131:40297)
0926 16:37:39.887786 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.887791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.888179 (+ 388us) server_negotiation.cc:407] Connection header received
0926 16:37:39.888389 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.888393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.888446 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.888511 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:39.888791 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.889539 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.891323 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.892009 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.892154 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.892222 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.892620 (+ 398us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.892703 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 16:37:39.892834 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.887592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:46931 (local address 127.11.158.130:46573)
0926 16:37:39.887724 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.887729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.887900 (+ 171us) server_negotiation.cc:407] Connection header received
0926 16:37:39.888083 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.888086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.888135 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.888221 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:39.888625 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.889424 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.891323 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.892018 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.892154 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.892222 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.892620 (+ 398us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.892703 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 16:37:39.892894 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.887483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:46573 (local address 127.11.158.129:46931)
0926 16:37:39.887799 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.887820 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.887956 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.888255 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.888261 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.888282 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:39.888465 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.888474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.889556 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.889559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.891162 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.891173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.892164 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.892168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.892413 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.892499 (+ 86us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.892582 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.892701 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":247,"thread_start_us":97,"threads_started":1}
I20260926 16:37:39.892892 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.887483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:40297 (local address 127.11.158.129:43609)
0926 16:37:39.888053 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:39.888069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:39.888229 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:39.888552 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:39.888558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:39.888580 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:39.888674 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.888681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.889659 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.889662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.891166 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:39.891173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.892164 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.892168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:39.892412 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.892502 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 16:37:39.892582 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:37:39.892701 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":626,"thread_start_us":97,"threads_started":1}
I20260926 16:37:39.893616 19069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc1ee39e1043deba4a636d4355d79b" candidate_uuid: "676b8797be7847339c9336fe0e2b191f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d5331360dd240e09eb6b57453ede29b" is_pre_election: true
I20260926 16:37:39.893819 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc1ee39e1043deba4a636d4355d79b" candidate_uuid: "676b8797be7847339c9336fe0e2b191f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" is_pre_election: true
I20260926 16:37:39.893931 19069 raft_consensus.cc:2454] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676b8797be7847339c9336fe0e2b191f in term 0.
I20260926 16:37:39.894110 19205 raft_consensus.cc:2454] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676b8797be7847339c9336fe0e2b191f in term 0.
I20260926 16:37:39.894505 18866 leader_election.cc:302] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d5331360dd240e09eb6b57453ede29b, 676b8797be7847339c9336fe0e2b191f; no voters:
I20260926 16:37:39.894773 19291 raft_consensus.cc:2781] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:39.894866 19291 raft_consensus.cc:482] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:39.894902 19291 raft_consensus.cc:3037] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:39.896039 19291 raft_consensus.cc:504] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.896442 19291 leader_election.cc:288] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [CANDIDATE]: Term 1 election: Requested vote from peers 5d5331360dd240e09eb6b57453ede29b (127.11.158.130:46573), f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297)
I20260926 16:37:39.896839 19069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc1ee39e1043deba4a636d4355d79b" candidate_uuid: "676b8797be7847339c9336fe0e2b191f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d5331360dd240e09eb6b57453ede29b"
I20260926 16:37:39.897007 19069 raft_consensus.cc:3037] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:39.898084 19069 raft_consensus.cc:2454] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676b8797be7847339c9336fe0e2b191f in term 1.
I20260926 16:37:39.898401 18866 leader_election.cc:302] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d5331360dd240e09eb6b57453ede29b, 676b8797be7847339c9336fe0e2b191f; no voters:
I20260926 16:37:39.898371 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fc1ee39e1043deba4a636d4355d79b" candidate_uuid: "676b8797be7847339c9336fe0e2b191f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4"
I20260926 16:37:39.898577 19291 raft_consensus.cc:2781] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:39.898555 19205 raft_consensus.cc:3037] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:39.898773 19291 raft_consensus.cc:686] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 1 LEADER]: Becoming Leader. State: Replica: 676b8797be7847339c9336fe0e2b191f, State: Running, Role: LEADER
I20260926 16:37:39.899147 19291 consensus_queue.cc:237] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:39.899827 19205 raft_consensus.cc:2454] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676b8797be7847339c9336fe0e2b191f in term 1.
I20260926 16:37:39.902887 18015 catalog_manager.cc:5495] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f reported cstate change: term changed from 0 to 1, leader changed from <none> to 676b8797be7847339c9336fe0e2b191f (127.11.158.129). New cstate: current_term: 1 leader_uuid: "676b8797be7847339c9336fe0e2b191f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } health_report { overall_health: HEALTHY } } }
I20260926 16:37:39.935680 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.932060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.11.158.129:43867)
0926 16:37:39.932189 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.932192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.932215 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:37:39.932289 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.932291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.932405 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.932508 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:39.932765 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.933280 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.934070 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.934341 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.934456 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.934804 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:39.935288 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:39.935420 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.935505 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.935570 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:37:39.941309 19205 raft_consensus.cc:1261] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Refusing update from remote peer 676b8797be7847339c9336fe0e2b191f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:39.941309 19069 raft_consensus.cc:1261] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 1 FOLLOWER]: Refusing update from remote peer 676b8797be7847339c9336fe0e2b191f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:39.942013 19291 consensus_queue.cc:1035] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:39.942298 19296 consensus_queue.cc:1035] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:39.949967 19299 mvcc.cc:205] Tried to move back new op lower bound from 7333644943112531968 to 7333644942949748736. Current Snapshot: MvccSnapshot[applied={T|T < 7333644943112531968}]
I20260926 16:37:39.951781 19301 mvcc.cc:205] Tried to move back new op lower bound from 7333644943112531968 to 7333644942949748736. Current Snapshot: MvccSnapshot[applied={T|T < 7333644943112531968}]
I20260926 16:37:39.952219 19300 mvcc.cc:205] Tried to move back new op lower bound from 7333644943112531968 to 7333644942949748736. Current Snapshot: MvccSnapshot[applied={T|T < 7333644943112531968}]
I20260926 16:37:39.963213 18015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:39.963368 18015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:39.967793 18015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b5b1419a34348c4814c9e5f9e9bcbf8: Sending DeleteTablet for 3 replicas of tablet 10fc1ee39e1043deba4a636d4355d79b
I20260926 16:37:39.968752 19185 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fc1ee39e1043deba4a636d4355d79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:39 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:54246
I20260926 16:37:39.969050 19049 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fc1ee39e1043deba4a636d4355d79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:39 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:32896
I20260926 16:37:39.969347 19308 tablet_replica.cc:330] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: stopping tablet replica
I20260926 16:37:39.969504 19309 tablet_replica.cc:330] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: stopping tablet replica
I20260926 16:37:39.969851 19308 raft_consensus.cc:2229] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:39.969899 19309 raft_consensus.cc:2229] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:39.970263 18912 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fc1ee39e1043deba4a636d4355d79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:39 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:52602
I20260926 16:37:39.970331 19309 raft_consensus.cc:2258] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:39.970330 19308 raft_consensus.cc:2258] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:39.971997 19308 ts_tablet_manager.cc:1905] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:39.972409 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 17983
I20260926 16:37:39.972707 19309 ts_tablet_manager.cc:1905] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:39.975133 19308 ts_tablet_manager.cc:1918] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:39.975310 19308 log.cc:1196] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/10fc1ee39e1043deba4a636d4355d79b
I20260926 16:37:39.975701 19308 ts_tablet_manager.cc:1939] T 10fc1ee39e1043deba4a636d4355d79b P f1bf0a9157a74f05a449cd80a6bf4af4: Deleting consensus metadata
I20260926 16:37:39.976126 19309 ts_tablet_manager.cc:1918] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:39.976231 19309 log.cc:1196] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/10fc1ee39e1043deba4a636d4355d79b
I20260926 16:37:39.976606 19309 ts_tablet_manager.cc:1939] T 10fc1ee39e1043deba4a636d4355d79b P 5d5331360dd240e09eb6b57453ede29b: Deleting consensus metadata
W20260926 16:37:39.980391 19005 connection.cc:535] server connection from 127.0.0.1:32896 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 16:37:39.980826 19139 connection.cc:535] server connection from 127.0.0.1:54246 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 16:37:39.981945 19310 tablet_replica.cc:330] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: stopping tablet replica
I20260926 16:37:39.982335 19310 raft_consensus.cc:2229] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:39.982852 19310 raft_consensus.cc:2258] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:39.984457 19310 ts_tablet_manager.cc:1905] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:39.987704 19310 ts_tablet_manager.cc:1918] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:39.987818 19310 log.cc:1196] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/10fc1ee39e1043deba4a636d4355d79b
I20260926 16:37:39.988196 19310 ts_tablet_manager.cc:1939] T 10fc1ee39e1043deba4a636d4355d79b P 676b8797be7847339c9336fe0e2b191f: Deleting consensus metadata
I20260926 16:37:39.988279 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.971707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42246 (local address 127.11.158.189:46079)
0926 16:37:39.971821 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.971825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.977578 (+ 5753us) server_negotiation.cc:407] Connection header received
0926 16:37:39.977784 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.977787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.977844 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.977897 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.978930 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.979773 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.981191 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.981553 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.981630 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.984219 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.984223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.984226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.984258 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.986122 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.986619 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.986623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.986624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.986742 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.987015 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.987018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.987020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.987273 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.987399 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.987933 (+ 534us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.988067 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260926 16:37:39.990383 18867 connection.cc:441] server connection from 127.0.0.1:52602 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52602 (request call id 1) could send its response
I20260926 16:37:39.998188 19266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.971951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.11.158.188:45501)
0926 16:37:39.972063 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.972069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.989057 (+ 16988us) server_negotiation.cc:407] Connection header received
0926 16:37:39.989104 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.989108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.989161 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.989232 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.990093 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.990819 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.991633 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.991865 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.991981 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.994016 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.994022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.994025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.994062 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.995997 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.996539 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.996545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.996548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.996651 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.996957 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.996961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.996963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.997181 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.997301 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.997888 (+ 587us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.998017 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:37:39.998631 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.985231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.11.158.189:46079)
0926 16:37:39.985340 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.985344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.987319 (+ 1975us) server_negotiation.cc:407] Connection header received
0926 16:37:39.987482 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.987485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.987530 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.987577 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.990299 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.991058 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.992401 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.992734 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.992810 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.994578 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.994582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.994585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.994616 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.996756 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.997232 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.997238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.997242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.997324 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.997610 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.997613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.997615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.997829 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.997924 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:39.998375 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 16:37:39.998476 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:37:40.000722 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:39.985842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36000 (local address 127.11.158.188:45501)
0926 16:37:39.985953 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:39.985957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:39.988083 (+ 2126us) server_negotiation.cc:407] Connection header received
0926 16:37:39.989298 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:39.989302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:39.989349 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:39.989400 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:39.992385 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.993200 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.994331 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:39.994675 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:39.994750 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:39.996355 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:39.996359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:39.996362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:39.996387 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:39.997969 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.999249 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.999253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.999255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.999334 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:39.999616 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:39.999619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:39.999621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:39.999817 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:39.999910 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:40.000239 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 16:37:40.000326 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:37:40.954697 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:40.954113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.131:45293)
0926 16:37:40.954449 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:40.954561 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":95,"threads_started":1}
I20260926 16:37:40.955271 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:40.954595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:48779)
0926 16:37:40.955093 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:40.955174 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":58,"threads_started":1}
W20260926 16:37:40.955524 19252 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:32879 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
W20260926 16:37:40.955636 19116 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:32879 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
I20260926 16:37:40.956327 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:40.954877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.129:43641)
0926 16:37:40.956026 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:40.956134 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":67,"threads_started":1}
W20260926 16:37:40.956732 18979 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:32879 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
I20260926 16:37:41.468128 19322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b5b1419a34348c4814c9e5f9e9bcbf8)
I20260926 16:37:41.468289 19322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } }
I20260926 16:37:41.468889 19322 leader_election.cc:288] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879), 78cfcea6928e460485d6300e5b396d80 (127.11.158.188:45501)
I20260926 16:37:41.469281 19324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b5b1419a34348c4814c9e5f9e9bcbf8)
I20260926 16:37:41.469489 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.469046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51092)
0926 16:37:41.469264 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.469348 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260926 16:37:41.469465 19324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } }
W20260926 16:37:41.469673 18603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
I20260926 16:37:41.470554 19324 leader_election.cc:288] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879), 0e8c28627ef94c9d824564d7b1fba272 (127.11.158.189:46079)
I20260926 16:37:41.470945 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.470600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51100)
0926 16:37:41.470778 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.470849 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":58,"threads_started":1}
W20260926 16:37:41.471095 18785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
I20260926 16:37:41.471374 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.470283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51098)
0926 16:37:41.470442 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.471299 (+ 857us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260926 16:37:41.471485 18603 leader_election.cc:334] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879): Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
I20260926 16:37:41.472141 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.471684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51114)
0926 16:37:41.472017 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.472069 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280}
W20260926 16:37:41.472257 18785 leader_election.cc:334] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879): Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
I20260926 16:37:41.473906 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.469365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:45501 (local address 127.0.0.1:58344)
0926 16:37:41.469615 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.469628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:41.469718 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:41.470471 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:41.470474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:41.470491 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:41.470598 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.470603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.471665 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.471669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.472879 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.472888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.473520 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.473523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.473611 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.473635 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:37:41.473717 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 16:37:41.473798 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":57,"threads_started":1}
I20260926 16:37:41.473955 19326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.469569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.11.158.188:45501)
0926 16:37:41.470235 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.470240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.470253 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:41.470300 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.470303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.470353 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.470449 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:41.470752 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.471547 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.472995 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.473406 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.473496 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.473535 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.473735 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.473829 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":63,"threads_started":1}
I20260926 16:37:41.474370 18809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e8c28627ef94c9d824564d7b1fba272" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "78cfcea6928e460485d6300e5b396d80" is_pre_election: true
I20260926 16:37:41.474563 18809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e8c28627ef94c9d824564d7b1fba272 in term 1.
I20260926 16:37:41.474918 18603 leader_election.cc:302] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e8c28627ef94c9d824564d7b1fba272, 78cfcea6928e460485d6300e5b396d80; no voters: 9b5b1419a34348c4814c9e5f9e9bcbf8
I20260926 16:37:41.475060 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.470887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:46079 (local address 127.0.0.1:51892)
0926 16:37:41.471062 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.471075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:41.471155 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:41.471738 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:41.471741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:41.471758 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:41.471832 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.471837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.472711 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.472715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.473681 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.473692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.474625 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.474628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.474753 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.474771 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:37:41.474865 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 16:37:41.474935 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260926 16:37:41.475092 19322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 16:37:41.475163 19322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9b5b1419a34348c4814c9e5f9e9bcbf8)
I20260926 16:37:41.475214 19322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:37:41.476195 19322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } }
I20260926 16:37:41.476414 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.470968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.11.158.189:46079)
0926 16:37:41.471315 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.471323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.471336 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:41.471373 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.471375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.471419 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.471647 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:41.471943 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.472601 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.473795 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.474516 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.474650 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.474694 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.474891 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.474954 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":91,"threads_started":1}
I20260926 16:37:41.476567 19322 leader_election.cc:288] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 election: Requested vote from peers 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879), 78cfcea6928e460485d6300e5b396d80 (127.11.158.188:45501)
I20260926 16:37:41.476877 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.476658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51120)
0926 16:37:41.476768 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.476820 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 16:37:41.476876 18627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78cfcea6928e460485d6300e5b396d80" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0e8c28627ef94c9d824564d7b1fba272" is_pre_election: true
I20260926 16:37:41.477023 18809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0e8c28627ef94c9d824564d7b1fba272" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "78cfcea6928e460485d6300e5b396d80"
I20260926 16:37:41.477115 18627 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 78cfcea6928e460485d6300e5b396d80 in current term 2: Already voted for candidate 0e8c28627ef94c9d824564d7b1fba272 in this term.
I20260926 16:37:41.477128 18809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:37:41.477489 18785 leader_election.cc:302] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78cfcea6928e460485d6300e5b396d80; no voters: 0e8c28627ef94c9d824564d7b1fba272, 9b5b1419a34348c4814c9e5f9e9bcbf8
I20260926 16:37:41.478186 18809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e8c28627ef94c9d824564d7b1fba272 in term 2.
I20260926 16:37:41.478296 19324 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:37:41.478514 18603 leader_election.cc:302] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e8c28627ef94c9d824564d7b1fba272, 78cfcea6928e460485d6300e5b396d80; no voters:
I20260926 16:37:41.478693 19322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:37:41.478885 19322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [term 2 LEADER]: Becoming Leader. State: Replica: 0e8c28627ef94c9d824564d7b1fba272, State: Running, Role: LEADER
I20260926 16:37:41.479120 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.478911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51134)
0926 16:37:41.479021 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.479066 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 16:37:41.479141 19322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } }
W20260926 16:37:41.479334 18603 leader_election.cc:334] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879): Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
I20260926 16:37:41.480162 19332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e8c28627ef94c9d824564d7b1fba272. Latest consensus state: current_term: 2 leader_uuid: "0e8c28627ef94c9d824564d7b1fba272" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:41.480247 19332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:41.480528 19334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:41.482723 19334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:41.483053 19334 catalog_manager.cc:1259] Loaded cluster ID: 0d6648a44d70470890f55acda11ed1ab
I20260926 16:37:41.483094 19334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:41.483551 19334 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:41.483886 19334 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Loaded TSK: 0
I20260926 16:37:41.484550 19334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:37:41.533299 18616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:41.536722 18616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:41.539952 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.539680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51154)
0926 16:37:41.539806 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.539868 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 16:37:41.539935 18809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [term 2 FOLLOWER]: Refusing update from remote peer 0e8c28627ef94c9d824564d7b1fba272: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 16:37:41.540439 19332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260926 16:37:41.540755 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.540532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.0.0.1:51158)
0926 16:37:41.540645 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.540696 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 16:37:41.540885 18603 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 -> Peer 9b5b1419a34348c4814c9e5f9e9bcbf8 (127.11.158.190:32879): Couldn't send request to peer 9b5b1419a34348c4814c9e5f9e9bcbf8. Status: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 16:37:41.542027 19324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e8c28627ef94c9d824564d7b1fba272. Latest consensus state: current_term: 2 leader_uuid: "0e8c28627ef94c9d824564d7b1fba272" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:41.542165 19324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:37:41.543349 19322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0e8c28627ef94c9d824564d7b1fba272" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:41.543493 19322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:41.543656 19336 mvcc.cc:205] Tried to move back new op lower bound from 7333644949662109696 to 7333644949420933120. Current Snapshot: MvccSnapshot[applied={T|T < 7333644949662109696}]
I20260926 16:37:41.543802 19324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "0e8c28627ef94c9d824564d7b1fba272" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b5b1419a34348c4814c9e5f9e9bcbf8" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 32879 } } peers { permanent_uuid: "0e8c28627ef94c9d824564d7b1fba272" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 46079 } attrs { promote: false } } peers { permanent_uuid: "78cfcea6928e460485d6300e5b396d80" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 45501 } attrs { promote: false } } }
I20260926 16:37:41.543915 19324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cfcea6928e460485d6300e5b396d80 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:37:41.554014 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.549586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:46573 (local address 127.0.0.1:40188)
0926 16:37:41.549699 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.549713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:41.549791 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:41.550785 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:41.550788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:41.550805 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:41.550898 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.550904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.551909 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.551913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.552994 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.553002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.553630 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.553633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.553719 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.553742 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:37:41.553815 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 16:37:41.553885 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 16:37:41.554036 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.549824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.11.158.130:46573)
0926 16:37:41.550576 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.550580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.550593 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:41.550644 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.550646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.550682 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.550780 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:41.551055 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.551791 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.553109 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.553518 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.553645 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.553698 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.553840 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.553910 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":76,"threads_started":1}
I20260926 16:37:41.554204 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.550151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43867 (local address 127.0.0.1:43734)
0926 16:37:41.550262 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.550276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:41.550361 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:41.551045 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:41.551049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:41.551069 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:41.551174 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.551179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.552254 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.552258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.553195 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.553203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.553827 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.553830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.553924 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.553946 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:37:41.554014 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 16:37:41.554077 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":37}
I20260926 16:37:41.554593 19049 tablet_service.cc:1467] Processing CreateTablet for tablet 99a40de48d2f4471b68196773e98019e (DEFAULT_TABLE table=test-table [id=8262f79b64d2462a9e47763dd60070dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:41.554883 19049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99a40de48d2f4471b68196773e98019e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:41.554986 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.550300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.11.158.129:43867)
0926 16:37:41.550797 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.550802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.550814 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:37:41.550860 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.550862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.550912 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.551281 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:41.551387 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.552114 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.553306 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.553709 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.553840 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.553893 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.554773 (+ 880us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.554851 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":74,"threads_started":1}
I20260926 16:37:41.555487 18912 tablet_service.cc:1467] Processing CreateTablet for tablet 99a40de48d2f4471b68196773e98019e (DEFAULT_TABLE table=test-table [id=8262f79b64d2462a9e47763dd60070dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:41.555728 18912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99a40de48d2f4471b68196773e98019e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:41.556494 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.550172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:40297 (local address 127.0.0.1:48808)
0926 16:37:41.550365 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.550378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:41.550467 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:41.550992 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:41.550997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:41.551015 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:41.551102 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.551107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.552265 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.552268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.554912 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.554921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.556118 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.556121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.556206 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.556227 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:37:41.556293 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 16:37:41.556379 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1479,"thread_start_us":81,"threads_started":1}
I20260926 16:37:41.556896 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.550523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.11.158.131:40297)
0926 16:37:41.550746 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.550751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.550765 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:37:41.550834 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.550839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.550883 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.550975 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:41.551212 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.551987 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.555290 (+ 3303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.555861 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.556002 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.556055 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.556682 (+ 627us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.556759 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260926 16:37:41.557369 19185 tablet_service.cc:1467] Processing CreateTablet for tablet 99a40de48d2f4471b68196773e98019e (DEFAULT_TABLE table=test-table [id=8262f79b64d2462a9e47763dd60070dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:41.557607 19185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99a40de48d2f4471b68196773e98019e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:41.557744 19345 tablet_bootstrap.cc:492] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: Bootstrap starting.
I20260926 16:37:41.558382 19346 tablet_bootstrap.cc:492] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f: Bootstrap starting.
I20260926 16:37:41.558665 19345 tablet_bootstrap.cc:654] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:41.559280 19346 tablet_bootstrap.cc:654] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:41.560340 19345 tablet_bootstrap.cc:492] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: No bootstrap required, opened a new log
I20260926 16:37:41.560446 19345 ts_tablet_manager.cc:1397] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260926 16:37:41.560501 19349 tablet_bootstrap.cc:492] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: Bootstrap starting.
I20260926 16:37:41.560545 19346 tablet_bootstrap.cc:492] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f: No bootstrap required, opened a new log
I20260926 16:37:41.560626 19346 ts_tablet_manager.cc:1397] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:41.560885 19345 raft_consensus.cc:348] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.561007 19345 raft_consensus.cc:374] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:41.560990 19346 raft_consensus.cc:348] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.561040 19345 raft_consensus.cc:729] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5331360dd240e09eb6b57453ede29b, State: Initialized, Role: FOLLOWER
I20260926 16:37:41.561069 19346 raft_consensus.cc:374] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:41.561090 19346 raft_consensus.cc:729] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676b8797be7847339c9336fe0e2b191f, State: Initialized, Role: FOLLOWER
I20260926 16:37:41.561164 19345 consensus_queue.cc:260] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.561225 19346 consensus_queue.cc:260] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.561360 19349 tablet_bootstrap.cc:654] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:41.561527 19345 ts_tablet_manager.cc:1428] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:37:41.561550 19346 ts_tablet_manager.cc:1428] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:41.563247 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.562602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:39015)
0926 16:37:41.563102 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.563158 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":85,"threads_started":1}
I20260926 16:37:41.564025 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.563825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:54979)
0926 16:37:41.563920 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.563967 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 16:37:41.564673 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.561952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.129:35603)
0926 16:37:41.562191 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.562254 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":112,"threads_started":1}
I20260926 16:37:41.565277 19117 heartbeater.cc:503] Master 127.11.158.189:46079 was elected leader, sending a full tablet report...
I20260926 16:37:41.565459 18980 heartbeater.cc:503] Master 127.11.158.189:46079 was elected leader, sending a full tablet report...
I20260926 16:37:41.565707 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.565440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.129:42071)
0926 16:37:41.565585 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.565633 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 16:37:41.566242 19349 tablet_bootstrap.cc:492] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: No bootstrap required, opened a new log
I20260926 16:37:41.566352 19349 ts_tablet_manager.cc:1397] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260926 16:37:41.566825 19349 raft_consensus.cc:348] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.566948 19349 raft_consensus.cc:374] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:41.566993 19349 raft_consensus.cc:729] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bf0a9157a74f05a449cd80a6bf4af4, State: Initialized, Role: FOLLOWER
I20260926 16:37:41.567126 19349 consensus_queue.cc:260] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.567469 19349 ts_tablet_manager.cc:1428] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:41.568089 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.567721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.131:59773)
0926 16:37:41.567927 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.567982 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260926 16:37:41.569195 19253 heartbeater.cc:503] Master 127.11.158.189:46079 was elected leader, sending a full tablet report...
I20260926 16:37:41.569334 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.569025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.131:44629)
0926 16:37:41.569131 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.569180 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 16:37:41.633960 19350 raft_consensus.cc:482] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:41.634184 19350 raft_consensus.cc:504] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.635531 19350 leader_election.cc:288] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297), 676b8797be7847339c9336fe0e2b191f (127.11.158.129:43867)
I20260926 16:37:41.640727 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.635749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:40297 (local address 127.11.158.130:40797)
0926 16:37:41.636006 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.636023 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:41.636156 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:41.636571 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:41.636574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:41.636609 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:41.636833 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.636838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.637992 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.637997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.639540 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.639551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.640241 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.640247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.640377 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.640416 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 16:37:41.640496 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:37:41.640557 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":132,"spinlock_wait_cycles":9984}
I20260926 16:37:41.640740 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.635865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:40797 (local address 127.11.158.131:40297)
0926 16:37:41.636008 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.636014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.636095 (+ 81us) server_negotiation.cc:407] Connection header received
0926 16:37:41.636329 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.636358 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.636415 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.636528 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:41.637016 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.637833 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.639683 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.640088 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.640189 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.640231 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.640521 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.640619 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 16:37:41.640857 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.635878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:47251 (local address 127.11.158.129:43867)
0926 16:37:41.636027 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.636032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.636279 (+ 247us) server_negotiation.cc:407] Connection header received
0926 16:37:41.636366 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.636368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.636415 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.636515 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:41.637155 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.637909 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.639699 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.640176 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.640304 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.640382 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.640648 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.640733 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:37:41.641098 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.635752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43867 (local address 127.11.158.130:47251)
0926 16:37:41.636153 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.636165 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:41.636258 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:41.636571 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:41.636574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:41.636607 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:41.636829 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.636838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.638020 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.638024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.639540 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:41.639551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.640320 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.640323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:41.640483 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.640510 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:37:41.640611 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 16:37:41.640678 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":233,"spinlock_wait_cycles":4736,"thread_start_us":134,"threads_started":1}
I20260926 16:37:41.641443 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a40de48d2f4471b68196773e98019e" candidate_uuid: "5d5331360dd240e09eb6b57453ede29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" is_pre_election: true
I20260926 16:37:41.641633 19205 raft_consensus.cc:2454] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d5331360dd240e09eb6b57453ede29b in term 0.
I20260926 16:37:41.641695 18932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a40de48d2f4471b68196773e98019e" candidate_uuid: "5d5331360dd240e09eb6b57453ede29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676b8797be7847339c9336fe0e2b191f" is_pre_election: true
I20260926 16:37:41.641937 18932 raft_consensus.cc:2454] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d5331360dd240e09eb6b57453ede29b in term 0.
I20260926 16:37:41.642078 19002 leader_election.cc:302] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d5331360dd240e09eb6b57453ede29b, f1bf0a9157a74f05a449cd80a6bf4af4; no voters:
I20260926 16:37:41.642347 19350 raft_consensus.cc:2781] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:41.642434 19350 raft_consensus.cc:482] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:41.642486 19350 raft_consensus.cc:3037] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:41.643532 19350 raft_consensus.cc:504] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.643935 19350 leader_election.cc:288] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [CANDIDATE]: Term 1 election: Requested vote from peers f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297), 676b8797be7847339c9336fe0e2b191f (127.11.158.129:43867)
I20260926 16:37:41.644227 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a40de48d2f4471b68196773e98019e" candidate_uuid: "5d5331360dd240e09eb6b57453ede29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4"
I20260926 16:37:41.644320 18932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a40de48d2f4471b68196773e98019e" candidate_uuid: "5d5331360dd240e09eb6b57453ede29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676b8797be7847339c9336fe0e2b191f"
I20260926 16:37:41.644388 19205 raft_consensus.cc:3037] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:41.644466 18932 raft_consensus.cc:3037] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:41.645419 19205 raft_consensus.cc:2454] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d5331360dd240e09eb6b57453ede29b in term 1.
I20260926 16:37:41.645584 18932 raft_consensus.cc:2454] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d5331360dd240e09eb6b57453ede29b in term 1.
I20260926 16:37:41.645707 19002 leader_election.cc:302] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d5331360dd240e09eb6b57453ede29b, f1bf0a9157a74f05a449cd80a6bf4af4; no voters:
I20260926 16:37:41.645917 19350 raft_consensus.cc:2781] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:41.646149 19350 raft_consensus.cc:686] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 1 LEADER]: Becoming Leader. State: Replica: 5d5331360dd240e09eb6b57453ede29b, State: Running, Role: LEADER
I20260926 16:37:41.646416 19350 consensus_queue.cc:237] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } }
I20260926 16:37:41.646837 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.646626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:51515)
0926 16:37:41.646721 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.646773 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 16:37:41.648716 18616 catalog_manager.cc:5495] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d5331360dd240e09eb6b57453ede29b (127.11.158.130). New cstate: current_term: 1 leader_uuid: "5d5331360dd240e09eb6b57453ede29b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5331360dd240e09eb6b57453ede29b" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 46573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:41.676769 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.672821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.11.158.130:46573)
0926 16:37:41.672966 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:41.672971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:41.672987 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:41.673111 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:41.673114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:41.673174 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:41.673279 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:41.673498 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.674283 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.675136 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:41.675477 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:41.675588 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:41.675836 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:41.676402 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:41.676506 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:41.676585 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 16:37:41.676640 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:37:41.679672 19205 raft_consensus.cc:1261] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Refusing update from remote peer 5d5331360dd240e09eb6b57453ede29b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:41.679848 18932 raft_consensus.cc:1261] T 99a40de48d2f4471b68196773e98019e P 676b8797be7847339c9336fe0e2b191f [term 1 FOLLOWER]: Refusing update from remote peer 5d5331360dd240e09eb6b57453ede29b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:41.680275 19358 consensus_queue.cc:1035] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bf0a9157a74f05a449cd80a6bf4af4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 40297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:41.680495 19350 consensus_queue.cc:1035] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "676b8797be7847339c9336fe0e2b191f" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:41.683159 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.682940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.131:41507)
0926 16:37:41.683034 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.683091 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 16:37:41.683364 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.682722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.129:46493)
0926 16:37:41.683226 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.683286 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413}
I20260926 16:37:41.685770 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.685481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:50215)
0926 16:37:41.685632 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.685692 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 16:37:41.688500 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.686820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.131:50799)
0926 16:37:41.688368 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.688428 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1457}
I20260926 16:37:41.688591 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.688389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.130:58659)
0926 16:37:41.688473 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.688525 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 16:37:41.690230 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.689879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:32879 (local address 127.11.158.129:54167)
0926 16:37:41.690106 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.690158 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:32879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260926 16:37:41.696610 18617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:41.696776 18617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:41.700565 18617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e8c28627ef94c9d824564d7b1fba272: Sending DeleteTablet for 3 replicas of tablet 99a40de48d2f4471b68196773e98019e
I20260926 16:37:41.701426 19049 tablet_service.cc:1514] Processing DeleteTablet for tablet 99a40de48d2f4471b68196773e98019e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:41 UTC) from {username='oryx', principal='oryx/127.11.158.189@KRBTEST.COM'} at 127.0.0.1:40188
I20260926 16:37:41.701573 19185 tablet_service.cc:1514] Processing DeleteTablet for tablet 99a40de48d2f4471b68196773e98019e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:41 UTC) from {username='oryx', principal='oryx/127.11.158.189@KRBTEST.COM'} at 127.0.0.1:48808
I20260926 16:37:41.701843 19371 tablet_replica.cc:330] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: stopping tablet replica
I20260926 16:37:41.701908 19372 tablet_replica.cc:330] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: stopping tablet replica
I20260926 16:37:41.702037 19372 raft_consensus.cc:2229] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:41.702037 19371 raft_consensus.cc:2229] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:41.702175 19372 raft_consensus.cc:2258] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:41.702241 18912 tablet_service.cc:1514] Processing DeleteTablet for tablet 99a40de48d2f4471b68196773e98019e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:41 UTC) from {username='oryx', principal='oryx/127.11.158.189@KRBTEST.COM'} at 127.0.0.1:43734
I20260926 16:37:41.702581 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 18761
I20260926 16:37:41.702529 19371 raft_consensus.cc:2258] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:41.702934 19372 ts_tablet_manager.cc:1905] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:41.704926 19371 ts_tablet_manager.cc:1905] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:41.705371 19372 ts_tablet_manager.cc:1918] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:41.705436 19372 log.cc:1196] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/99a40de48d2f4471b68196773e98019e
I20260926 16:37:41.705710 19372 ts_tablet_manager.cc:1939] T 99a40de48d2f4471b68196773e98019e P f1bf0a9157a74f05a449cd80a6bf4af4: Deleting consensus metadata
I20260926 16:37:41.706475 18601 catalog_manager.cc:4841] TS f1bf0a9157a74f05a449cd80a6bf4af4 (127.11.158.131:40297): tablet 99a40de48d2f4471b68196773e98019e (table test-table [id=8262f79b64d2462a9e47763dd60070dd]) successfully deleted
I20260926 16:37:41.707585 19371 ts_tablet_manager.cc:1918] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:41.707688 19371 log.cc:1196] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/99a40de48d2f4471b68196773e98019e
I20260926 16:37:41.707957 19371 ts_tablet_manager.cc:1939] T 99a40de48d2f4471b68196773e98019e P 5d5331360dd240e09eb6b57453ede29b: Deleting consensus metadata
I20260926 16:37:41.708659 18602 catalog_manager.cc:4841] TS 5d5331360dd240e09eb6b57453ede29b (127.11.158.130:46573): tablet 99a40de48d2f4471b68196773e98019e (table test-table [id=8262f79b64d2462a9e47763dd60070dd]) successfully deleted
I20260926 16:37:41.714351 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 18986
I20260926 16:37:41.714376 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:41.714103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:43867 (local address 127.0.0.1:43746)
0926 16:37:41.714233 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:41.714302 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:43867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:37:41.715252 18603 catalog_manager.cc:4584] TS 676b8797be7847339c9336fe0e2b191f (127.11.158.129:43867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99a40de48d2f4471b68196773e98019e: Network error: Client connection negotiation failed: client connection to 127.11.158.129:43867: connect: Connection refused (error 111)
I20260926 16:37:41.720415 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 19122
I20260926 16:37:41.725878 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 18479
I20260926 16:37:41.731786 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 18583
2026-09-26T16:37:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5547 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:41 dist-test-slave-p39x krb5kdc[19376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:41 dist-test-slave-p39x krb5kdc[19376](info): set up 2 sockets
Sep 26 16:37:41 dist-test-slave-p39x krb5kdc[19376](info): commencing operation
krb5kdc: starting...
W20260926 16:37:42.809585 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:42 dist-test-slave-p39x krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab.
Entry for principal oryx/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab.
I20260926 16:37:42.935974 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34013
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab
--principal=oryx/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:34013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:43.045974 19389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:43.046371 19389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:43.046432 19389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:43.051523 19389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:43.051595 19389 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 16:37:43.051613 19389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:43.051632 19389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:43.051648 19389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:43.051664 19389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:43.057307 19389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:34013
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.190.keytab
--principal=oryx/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:43.058354 19389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:43.059335 19389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:43.068477 19397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.068477 19395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.068555 19394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:43.068671 19389 server_base.cc:1034] running on GCE node
I20260926 16:37:43.069567 19389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:43.070889 19389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:43.072062 19389 hybrid_clock.cc:648] HybridClock initialized: now 1790440663072028 us; error 48 us; skew 500 ppm
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:43.075023 19389 init.cc:377] Logged in from keytab as oryx/127.11.158.190@KRBTEST.COM (short username oryx)
I20260926 16:37:43.075910 19389 webserver.cc:460] Webserver started at http://127.11.158.190:44157/ using document root <none> and password file <none>
I20260926 16:37:43.076529 19389 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:43.076610 19389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:43.076784 19389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:43.078852 19389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "472d537977394f2ab338d92a970a3cf3"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "a035cb858dd408aa6e62ea0eb4c4329c"
server_key_iv: "a814ae7f211e46d72a2b10a42bbd0357"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.079375 19389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "472d537977394f2ab338d92a970a3cf3"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "a035cb858dd408aa6e62ea0eb4c4329c"
server_key_iv: "a814ae7f211e46d72a2b10a42bbd0357"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.083359 19389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 16:37:43.086086 19404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.087608 19389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:37:43.087790 19389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "472d537977394f2ab338d92a970a3cf3"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "a035cb858dd408aa6e62ea0eb4c4329c"
server_key_iv: "a814ae7f211e46d72a2b10a42bbd0357"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.087931 19389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:43.101709 19389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:43.104976 19389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:43.105201 19389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:43.113346 19389 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:34013
I20260926 16:37:43.113399 19466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:34013 every 8 connection(s)
I20260926 16:37:43.114356 19389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:43.117525 19467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:43.122622 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 19389
I20260926 16:37:43.122742 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:43.123039 11898 external_mini_cluster.cc:1428] Setting key 8a1fe1afa7fe22804448c0249eee18b6
I20260926 16:37:43.123682 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Bootstrap starting.
I20260926 16:37:43.126416 19467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:43.127236 19467 log.cc:824] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:43.129338 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: No bootstrap required, opened a new log
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:43.133905 19467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472d537977394f2ab338d92a970a3cf3" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34013 } }
I20260926 16:37:43.134275 19467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:43.134368 19467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 472d537977394f2ab338d92a970a3cf3, State: Initialized, Role: FOLLOWER
I20260926 16:37:43.134923 19467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472d537977394f2ab338d92a970a3cf3" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34013 } }
I20260926 16:37:43.135074 19467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:43.135170 19467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:43.135285 19467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:43.136319 19467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472d537977394f2ab338d92a970a3cf3" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34013 } }
I20260926 16:37:43.136781 19467 leader_election.cc:302] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 472d537977394f2ab338d92a970a3cf3; no voters:
I20260926 16:37:43.137204 19467 leader_election.cc:288] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:43.137368 19472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:43.137655 19472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [term 1 LEADER]: Becoming Leader. State: Replica: 472d537977394f2ab338d92a970a3cf3, State: Running, Role: LEADER
I20260926 16:37:43.138041 19472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472d537977394f2ab338d92a970a3cf3" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34013 } }
I20260926 16:37:43.138561 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.124607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.11.158.190:34013)
0926 16:37:43.125099 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.125114 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.125151 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:37:43.125796 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.125814 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.126166 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.126517 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.127336 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.128416 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.129261 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.129817 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.129995 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.133026 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.133032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.133047 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.133079 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.135175 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.135729 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.135735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.135742 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.135870 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.136169 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.136173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.136175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.136736 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.136941 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.137257 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.137608 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":151,"threads_started":1}
I20260926 16:37:43.138680 19467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:43.139364 19472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 472d537977394f2ab338d92a970a3cf3. Latest consensus state: current_term: 1 leader_uuid: "472d537977394f2ab338d92a970a3cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472d537977394f2ab338d92a970a3cf3" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34013 } } }
I20260926 16:37:43.139490 19472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:43.139791 19473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "472d537977394f2ab338d92a970a3cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472d537977394f2ab338d92a970a3cf3" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34013 } } }
I20260926 16:37:43.139885 19473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3 [sys.catalog]: This master's current role is: LEADER
W20260926 16:37:43.143848 19487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:37:43.143981 19487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:37:43.144069 19478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:43.144961 19478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:43.151140 19478 catalog_manager.cc:1347] Generated new cluster ID: 0f3ebace97814d0a9808d3e83790616c
I20260926 16:37:43.151234 19478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:43.167325 19478 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:43.168710 19478 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:43.174801 19478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Generated new TSK 0
I20260926 16:37:43.175657 19478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab.
Entry for principal oryx/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab.
I20260926 16:37:43.211820 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34013
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab
--principal=oryx/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:43.327009 19493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:43.327369 19493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:43.327451 19493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:43.331576 19493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:43.331661 19493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:43.331705 19493 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:43.331785 19493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:43.337234 19493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab
--principal=oryx/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:34013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:43.338359 19493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:43.339519 19493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:43.349444 19501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.349502 19499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.349439 19498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:43.349754 19493 server_base.cc:1034] running on GCE node
I20260926 16:37:43.350206 19493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:43.350812 19493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:43.351995 19493 hybrid_clock.cc:648] HybridClock initialized: now 1790440663351983 us; error 38 us; skew 500 ppm
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:43.354856 19493 init.cc:377] Logged in from keytab as oryx/127.11.158.129@KRBTEST.COM (short username oryx)
I20260926 16:37:43.355726 19493 webserver.cc:460] Webserver started at http://127.11.158.129:44991/ using document root <none> and password file <none>
I20260926 16:37:43.356313 19493 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:43.356428 19493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:43.356683 19493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:43.358773 19493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "bb73f4a07e614ebe8b460b2cf486c022"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "0147bc0d740aea6e9cc0bf928dee597e"
server_key_iv: "39929234a0f49279dff2b96c7289e6ed"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.359344 19493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "bb73f4a07e614ebe8b460b2cf486c022"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "0147bc0d740aea6e9cc0bf928dee597e"
server_key_iv: "39929234a0f49279dff2b96c7289e6ed"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.363148 19493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:37:43.365789 19508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.367010 19493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:43.367112 19493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "bb73f4a07e614ebe8b460b2cf486c022"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "0147bc0d740aea6e9cc0bf928dee597e"
server_key_iv: "39929234a0f49279dff2b96c7289e6ed"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.367247 19493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:43.401587 19493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:43.404687 19493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:43.404889 19493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:43.406337 19493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:43.406385 19493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.406450 19493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:43.406481 19493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:43.417037 19493 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:36449
I20260926 16:37:43.417919 19493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:43.418846 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 19493
I20260926 16:37:43.418985 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:43.419270 11898 external_mini_cluster.cc:1428] Setting key 2b6d96275e20c044b6ea95b8a7c47354
I20260926 16:37:43.420365 19623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:36449 every 8 connection(s)
I20260926 16:37:43.422443 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.407327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:34921 (local address 127.11.158.190:34013)
0926 16:37:43.407481 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.407487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.408362 (+ 875us) server_negotiation.cc:407] Connection header received
0926 16:37:43.409498 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.409501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.409559 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.409669 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.411298 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.412172 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.414097 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.414328 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.414419 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.417610 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.417614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.417617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.417653 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.420179 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.420824 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.420827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.420829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.420910 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.421292 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.421294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.421296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.421468 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.421541 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.422181 (+ 640us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.422290 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:37:43.423385 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.407717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34013 (local address 127.11.158.129:34921)
0926 16:37:43.408177 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.408227 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.409249 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.409851 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.409862 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.410351 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:43.411081 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.411102 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.412378 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.412382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.413963 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.413975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.414494 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.414500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.414738 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.415481 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:43.415507 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:43.417423 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:43.420305 (+ 2882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.420313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.420330 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.420728 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.421003 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.421006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.421007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.421191 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.421578 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:43.421588 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:43.421916 (+ 328us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.422126 (+ 210us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.422498 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":139,"threads_started":1}
I20260926 16:37:43.424680 19624 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34013
I20260926 16:37:43.424927 19624 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:43.425510 19624 heartbeater.cc:511] Master 127.11.158.190:34013 requested a full tablet report, sending...
I20260926 16:37:43.427049 19420 ts_manager.cc:194] Registered new tserver with Master: bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449)
I20260926 16:37:43.428537 19420 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:34921
WARNING: no policy specified for oryx/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.130@KRBTEST.COM" created.
I20260926 16:37:43.437139 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.427594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:43087 (local address 127.11.158.190:34013)
0926 16:37:43.427777 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.427782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.430117 (+ 2335us) server_negotiation.cc:407] Connection header received
0926 16:37:43.430285 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.430288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.430336 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.430421 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.430886 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.431394 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.432219 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.432493 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.432603 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.433863 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.433866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.433868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.433890 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.435328 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.435829 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.435832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.435834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.435910 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.436228 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.436230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.436231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.436469 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.436554 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.436856 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.436984 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 16:37:43.437286 19631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.427521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34013 (local address 127.11.158.129:43087)
0926 16:37:43.430006 (+ 2485us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.430024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.430180 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.430438 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.430441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.430640 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:43.430765 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.430775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.431525 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.431528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.432080 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.432088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.432610 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.432613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.432705 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.433216 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:43.433238 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:43.433723 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:43.435440 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.435445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.435450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.435726 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.436007 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.436013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.436018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.436130 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.436563 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:43.436569 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:43.436685 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.436808 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.436985 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2360,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab.
Entry for principal oryx/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab.
I20260926 16:37:43.451130 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34013
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab
--principal=oryx/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:43.567289 19633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:43.567586 19633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:43.567634 19633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:43.571841 19633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:43.571916 19633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:43.571957 19633 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:43.571991 19633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:43.577947 19633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab
--principal=oryx/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:34013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:43.579078 19633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:43.580090 19633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:43.590415 19639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.590493 19638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.590436 19641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:43.590792 19633 server_base.cc:1034] running on GCE node
I20260926 16:37:43.591257 19633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:43.592097 19633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:43.593312 19633 hybrid_clock.cc:648] HybridClock initialized: now 1790440663593260 us; error 64 us; skew 500 ppm
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:43.596395 19633 init.cc:377] Logged in from keytab as oryx/127.11.158.130@KRBTEST.COM (short username oryx)
I20260926 16:37:43.597396 19633 webserver.cc:460] Webserver started at http://127.11.158.130:34191/ using document root <none> and password file <none>
I20260926 16:37:43.597990 19633 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:43.598084 19633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:43.598317 19633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:43.600493 19633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "6405f2545dca406984b16732bfcb7ce3"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "55bf09f3cd4f28c7d9737dadc384ef88"
server_key_iv: "e1583c41b74661349dca02efa9242d95"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.601082 19633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "6405f2545dca406984b16732bfcb7ce3"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "55bf09f3cd4f28c7d9737dadc384ef88"
server_key_iv: "e1583c41b74661349dca02efa9242d95"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.605026 19633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:37:43.607636 19648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.608971 19633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:43.609113 19633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "6405f2545dca406984b16732bfcb7ce3"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "55bf09f3cd4f28c7d9737dadc384ef88"
server_key_iv: "e1583c41b74661349dca02efa9242d95"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.609236 19633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:43.630450 19633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:43.633910 19633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:43.634131 19633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:43.635594 19633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:43.635675 19633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.635751 19633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:43.635795 19633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:43.647377 19633 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:39367
I20260926 16:37:43.647445 19763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:39367 every 8 connection(s)
I20260926 16:37:43.648521 19633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:43.651556 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.636752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:34193 (local address 127.11.158.190:34013)
0926 16:37:43.636913 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.636919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.637896 (+ 977us) server_negotiation.cc:407] Connection header received
0926 16:37:43.639262 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.639267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.639335 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.639432 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.641383 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.642278 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.644281 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.644604 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.644709 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.648011 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.648020 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.648034 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.648062 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.649789 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.650267 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.650270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.650271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.650346 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.650625 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.650627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.650628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.650803 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.650877 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.651329 (+ 452us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.651429 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 16:37:43.652437 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.637223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34013 (local address 127.11.158.130:34193)
0926 16:37:43.637712 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.637782 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.639026 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.639677 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.639689 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.640214 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:43.641128 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.641154 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.642448 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.642453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.644137 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.644149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.644780 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.644785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.645029 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.645819 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:43.645847 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:43.647815 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:43.649899 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.649903 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.649915 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.650175 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.650424 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.650426 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.650427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.650549 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.650885 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:43.650890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:43.651111 (+ 221us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.651283 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.651529 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":155,"threads_started":1}
I20260926 16:37:43.653712 19764 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34013
I20260926 16:37:43.653875 19764 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:43.654456 19764 heartbeater.cc:511] Master 127.11.158.190:34013 requested a full tablet report, sending...
I20260926 16:37:43.655534 19421 ts_manager.cc:194] Registered new tserver with Master: 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367)
I20260926 16:37:43.656213 19421 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:34193
I20260926 16:37:43.658329 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 19633
I20260926 16:37:43.658479 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:43.658756 11898 external_mini_cluster.cc:1428] Setting key 7f9523d9e76502edf3595787e9aec5a2
I20260926 16:37:43.663246 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.655791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34013 (local address 127.11.158.130:38717)
0926 16:37:43.656133 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.656144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.656255 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.656535 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.656538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.656746 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:43.656839 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.656844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.657582 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.657586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.658164 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.658172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.658746 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.658750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.658840 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.659376 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:43.659387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:43.659701 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:43.661445 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.661450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.661452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.661660 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.661973 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.661976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.661977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.662065 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.662599 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:43.662601 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:43.662698 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.662787 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.663082 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":108,"threads_started":1}
I20260926 16:37:43.663246 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.655857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:38717 (local address 127.11.158.190:34013)
0926 16:37:43.655990 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.655995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.656214 (+ 219us) server_negotiation.cc:407] Connection header received
0926 16:37:43.656379 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.656384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.656428 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.656520 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.656950 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.657458 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.658281 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.658627 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.658745 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.659832 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.659835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.659838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.659861 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.661325 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.661759 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.661765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.661769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.661863 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.662160 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.662165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.662177 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.662485 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.662603 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.662875 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.663063 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for oryx/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab.
Entry for principal oryx/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab.
I20260926 16:37:43.689203 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34013
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab
--principal=oryx/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:40289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:43.805589 19773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:43.805939 19773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:43.806025 19773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:43.810257 19773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:43.810376 19773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:43.810456 19773 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:37:43.810551 19773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:43.816219 19773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab
--principal=oryx/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.158.190:34013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:43.817407 19773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:43.818446 19773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:43.828502 19778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.828470 19779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:43.828470 19781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:43.828878 19773 server_base.cc:1034] running on GCE node
I20260926 16:37:43.829276 19773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:43.829952 19773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:43.831135 19773 hybrid_clock.cc:648] HybridClock initialized: now 1790440663831097 us; error 52 us; skew 500 ppm
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:43.834077 19773 init.cc:377] Logged in from keytab as oryx/127.11.158.131@KRBTEST.COM (short username oryx)
I20260926 16:37:43.835042 19773 webserver.cc:460] Webserver started at http://127.11.158.131:40223/ using document root <none> and password file <none>
I20260926 16:37:43.835629 19773 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:43.835698 19773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:43.835919 19773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:43.838081 19773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "25fc282343554c8bab9711ee90f0b074"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "301e333c3bda7596a6c4d305fbb94337"
server_key_iv: "b21b5e229d07fe361c78c4abd6b2ca8c"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.838702 19773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "25fc282343554c8bab9711ee90f0b074"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "301e333c3bda7596a6c4d305fbb94337"
server_key_iv: "b21b5e229d07fe361c78c4abd6b2ca8c"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.843039 19773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260926 16:37:43.846442 19788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.847996 19773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:43.848152 19773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "25fc282343554c8bab9711ee90f0b074"
format_stamp: "Formatted at 2026-09-26 16:37:43 on dist-test-slave-p39x"
server_key: "301e333c3bda7596a6c4d305fbb94337"
server_key_iv: "b21b5e229d07fe361c78c4abd6b2ca8c"
server_key_version: "encryptionkey@0"
I20260926 16:37:43.848296 19773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:43.861141 19773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:43.864312 19773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:43.864579 19773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:43.866142 19773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:43.866216 19773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.866286 19773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:43.866333 19773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:43.876926 19773 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:33005
I20260926 16:37:43.877038 19903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:33005 every 8 connection(s)
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:43.878067 19773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:43.881615 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.867195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:41755 (local address 127.11.158.190:34013)
0926 16:37:43.867327 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.867331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.868458 (+ 1127us) server_negotiation.cc:407] Connection header received
0926 16:37:43.869725 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.869729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.869789 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.869875 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.871609 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.872577 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.874270 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.874673 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.874812 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.877986 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.877989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.877991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.878022 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.879703 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.880226 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.880229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.880230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.880306 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.880660 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.880662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.880663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.880851 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.880928 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.881383 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.881484 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:43.882508 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.867776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34013 (local address 127.11.158.131:41755)
0926 16:37:43.868258 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.868300 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.869455 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.870111 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.870121 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.870686 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:43.871418 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.871437 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.872732 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.872738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.874119 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.874129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.874844 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.874849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.875076 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.875815 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:43.875836 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:43.877748 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:43.879838 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.879848 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.879860 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.880130 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.880432 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.880435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.880436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.880571 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.880939 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:43.880945 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:43.881181 (+ 236us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.881337 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.881612 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":136,"threads_started":1}
I20260926 16:37:43.883877 19904 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34013
I20260926 16:37:43.884061 19904 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:43.884698 19904 heartbeater.cc:511] Master 127.11.158.190:34013 requested a full tablet report, sending...
I20260926 16:37:43.885903 19420 ts_manager.cc:194] Registered new tserver with Master: 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:43.886258 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 19773
I20260926 16:37:43.886397 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:43.886763 11898 external_mini_cluster.cc:1428] Setting key 1a34191611f05fbc8ceef92fd193691d
I20260926 16:37:43.886749 19420 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:41755
I20260926 16:37:43.888890 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:37:43.894137 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.886369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:36309 (local address 127.11.158.190:34013)
0926 16:37:43.886476 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.886480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.886557 (+ 77us) server_negotiation.cc:407] Connection header received
0926 16:37:43.886687 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.886690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.886737 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.886825 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.887402 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.888077 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.889097 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.889327 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.889437 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.890571 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.890575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.890578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.890604 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.892087 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.892642 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.892646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.892648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.892743 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.893077 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.893114 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.893118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.893353 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.893438 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.893892 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.894015 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:37:43.894186 19910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.886056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34013 (local address 127.11.158.131:36309)
0926 16:37:43.886460 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.886478 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.886590 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.886891 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.886894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.887128 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:43.887286 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.887293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.888208 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.888212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.888918 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.888926 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.889437 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.889440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.889534 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.890095 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:43.890106 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:43.890433 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:43.892227 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.892233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.892238 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.892534 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.892844 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:43.892850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:43.892855 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:43.892960 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:43.893459 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:43.893465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:43.893592 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.893717 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.893903 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":136,"threads_started":1}
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:37:43.914433 19470 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:43.907857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54584 (local address 127.11.158.190:34013)
0926 16:37:43.908004 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.908007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.908021 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:37:43.908064 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.908067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.908108 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.908186 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.908986 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.909514 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.910303 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.910508 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.910586 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.914278 (+ 3692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54584: BlockingRecv error: recv got EOF from 127.0.0.1:54584 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 16:37:43 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:43.924430 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.915793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.11.158.190:34013)
0926 16:37:43.915950 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.915955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.916090 (+ 135us) server_negotiation.cc:407] Connection header received
0926 16:37:43.916257 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.916265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.916306 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.916437 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:43.917060 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.917601 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.918403 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.918631 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.918736 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.921244 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:43.921248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:43.921250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:43.921279 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:43.922636 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.923092 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.923100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.923102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.923181 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:43.923469 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:43.923473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:43.923474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:43.923673 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:43.923838 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.924074 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.924198 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 16:37:43.927563 19420 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:43.930768 19420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:43.946385 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.940853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60914 (local address 127.11.158.129:36449)
0926 16:37:43.941811 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.941818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.941842 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:37:43.941923 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.941928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.942118 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.942280 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:43.942514 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.943695 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.945071 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.945673 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.945819 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.945960 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.946134 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.946232 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":84,"threads_started":1}
I20260926 16:37:43.946738 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.940104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36449 (local address 127.0.0.1:60914)
0926 16:37:43.941108 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.941162 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.941296 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.942257 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.942261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.942279 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:43.942390 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.942397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.943833 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.943838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.944934 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.944944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.945803 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.945807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.945913 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.945949 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.946102 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.946235 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260926 16:37:43.946852 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.940964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.11.158.131:33005)
0926 16:37:43.941596 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.941611 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.941630 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:37:43.941692 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.941696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.941845 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.941978 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:43.942483 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.943249 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.944940 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.945688 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.945828 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.945960 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.946612 (+ 652us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.946710 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":90,"threads_started":1}
I20260926 16:37:43.947016 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.940566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:33005 (local address 127.0.0.1:41884)
0926 16:37:43.941470 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.941485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.941564 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.942102 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.942104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.942133 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:43.942354 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.942362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.943371 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.943377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.944799 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.944811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.946029 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.946033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.946148 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.946179 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.946246 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.946322 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":44,"spinlock_wait_cycles":3840,"thread_start_us":100,"threads_started":1}
I20260926 16:37:43.946764 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.940282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39367 (local address 127.0.0.1:37148)
0926 16:37:43.941117 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:43.941162 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:43.941296 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:43.942095 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:43.942101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:43.942133 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:43.942360 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.942366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.943799 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.943802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.945139 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:43.945145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.946369 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.946372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:43.946465 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.946489 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:37:43.946564 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 16:37:43.946640 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":267,"spinlock_wait_cycles":14336,"thread_start_us":119,"threads_started":1}
I20260926 16:37:43.947117 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:43.940536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.11.158.130:39367)
0926 16:37:43.941002 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:43.941014 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:43.941438 (+ 424us) server_negotiation.cc:407] Connection header received
0926 16:37:43.941538 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:43.941544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:43.941764 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:43.941921 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:43.942514 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.943674 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.945259 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:43.946059 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:43.946188 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:43.946311 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:43.946863 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 16:37:43.946967 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":128,"threads_started":1}
I20260926 16:37:43.949838 19558 tablet_service.cc:1467] Processing CreateTablet for tablet 387d6dd75ce7497f9301cc02ea24ced4 (DEFAULT_TABLE table=test-table [id=a1862137fb134461af9ea5a9835622be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:43.949880 19821 tablet_service.cc:1467] Processing CreateTablet for tablet 387d6dd75ce7497f9301cc02ea24ced4 (DEFAULT_TABLE table=test-table [id=a1862137fb134461af9ea5a9835622be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:43.949880 19698 tablet_service.cc:1467] Processing CreateTablet for tablet 387d6dd75ce7497f9301cc02ea24ced4 (DEFAULT_TABLE table=test-table [id=a1862137fb134461af9ea5a9835622be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:43.950868 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387d6dd75ce7497f9301cc02ea24ced4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:43.950887 19698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387d6dd75ce7497f9301cc02ea24ced4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:43.950896 19821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387d6dd75ce7497f9301cc02ea24ced4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:43.959432 19934 tablet_bootstrap.cc:492] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Bootstrap starting.
I20260926 16:37:43.960055 19932 tablet_bootstrap.cc:492] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Bootstrap starting.
I20260926 16:37:43.960554 19933 tablet_bootstrap.cc:492] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Bootstrap starting.
I20260926 16:37:43.961700 19934 tablet_bootstrap.cc:654] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:43.962260 19932 tablet_bootstrap.cc:654] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:43.962551 19933 tablet_bootstrap.cc:654] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:43.962536 19934 log.cc:824] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:43.963135 19932 log.cc:824] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:43.963285 19933 log.cc:824] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:43.965348 19933 tablet_bootstrap.cc:492] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: No bootstrap required, opened a new log
I20260926 16:37:43.965400 19934 tablet_bootstrap.cc:492] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: No bootstrap required, opened a new log
I20260926 16:37:43.965399 19932 tablet_bootstrap.cc:492] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: No bootstrap required, opened a new log
I20260926 16:37:43.965582 19933 ts_tablet_manager.cc:1397] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:37:43.965582 19932 ts_tablet_manager.cc:1397] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:43.965582 19934 ts_tablet_manager.cc:1397] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:37:43.968731 19933 raft_consensus.cc:348] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:43.969020 19933 raft_consensus.cc:374] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:43.969082 19933 raft_consensus.cc:729] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6405f2545dca406984b16732bfcb7ce3, State: Initialized, Role: FOLLOWER
I20260926 16:37:43.969552 19933 consensus_queue.cc:260] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:43.969811 19934 raft_consensus.cc:348] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:43.970088 19934 raft_consensus.cc:374] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:43.970148 19934 raft_consensus.cc:729] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25fc282343554c8bab9711ee90f0b074, State: Initialized, Role: FOLLOWER
I20260926 16:37:43.970451 19764 heartbeater.cc:503] Master 127.11.158.190:34013 was elected leader, sending a full tablet report...
I20260926 16:37:43.970611 19934 consensus_queue.cc:260] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:43.970793 19933 ts_tablet_manager.cc:1428] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:37:43.970949 19932 raft_consensus.cc:348] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:43.971207 19932 raft_consensus.cc:374] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:43.971256 19932 raft_consensus.cc:729] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Initialized, Role: FOLLOWER
I20260926 16:37:43.971733 19934 ts_tablet_manager.cc:1428] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:37:43.971652 19932 consensus_queue.cc:260] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:43.972116 19904 heartbeater.cc:503] Master 127.11.158.190:34013 was elected leader, sending a full tablet report...
I20260926 16:37:43.972755 19624 heartbeater.cc:503] Master 127.11.158.190:34013 was elected leader, sending a full tablet report...
I20260926 16:37:43.973557 19932 ts_tablet_manager.cc:1428] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260926 16:37:44.080185 19938 raft_consensus.cc:482] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:44.080489 19938 raft_consensus.cc:504] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.081760 19938 leader_election.cc:288] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.087486 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.082022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36449 (local address 127.11.158.130:50511)
0926 16:37:44.082163 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.082191 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.082296 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.082657 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.082662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.082723 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:44.082911 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.082923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.084020 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.084026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.085580 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.085596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.086944 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.086947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.087115 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.087150 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.087238 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.087329 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"spinlock_wait_cycles":6400}
I20260926 16:37:44.087582 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.082139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:50511 (local address 127.11.158.129:36449)
0926 16:37:44.082256 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.082260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.082278 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:37:44.082446 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.082450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.082504 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.082596 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:44.083044 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.083888 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.085787 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.086665 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.086785 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.086876 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.087316 (+ 440us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.087449 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:37:44.088219 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.082147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:57399 (local address 127.11.158.131:33005)
0926 16:37:44.082302 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.082307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.082686 (+ 379us) server_negotiation.cc:407] Connection header received
0926 16:37:44.082952 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.082960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.083078 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.083171 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:44.083618 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.084399 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.086552 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.087144 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.087271 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.087320 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.088040 (+ 720us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.088116 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:37:44.088227 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.082034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:33005 (local address 127.11.158.130:57399)
0926 16:37:44.082606 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.082620 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.082764 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.083245 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.083251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.083275 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:44.083481 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.083493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.084514 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.084517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.086406 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.086417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.087783 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.087788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.087883 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.087907 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.088011 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.088081 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":163,"spinlock_wait_cycles":1408,"thread_start_us":154,"threads_started":1}
I20260926 16:37:44.088609 19578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387d6dd75ce7497f9301cc02ea24ced4" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb73f4a07e614ebe8b460b2cf486c022" is_pre_election: true
I20260926 16:37:44.089004 19578 raft_consensus.cc:2454] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 0.
I20260926 16:37:44.089550 19858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387d6dd75ce7497f9301cc02ea24ced4" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074" is_pre_election: true
I20260926 16:37:44.089617 19652 leader_election.cc:302] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6405f2545dca406984b16732bfcb7ce3, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.089823 19858 raft_consensus.cc:2454] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 0.
I20260926 16:37:44.089941 19938 raft_consensus.cc:2781] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:44.090025 19938 raft_consensus.cc:482] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:44.090224 19938 raft_consensus.cc:3037] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.091324 19938 raft_consensus.cc:504] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.091748 19938 leader_election.cc:288] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 election: Requested vote from peers bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.092273 19858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387d6dd75ce7497f9301cc02ea24ced4" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074"
I20260926 16:37:44.092276 19578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387d6dd75ce7497f9301cc02ea24ced4" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb73f4a07e614ebe8b460b2cf486c022"
I20260926 16:37:44.092442 19858 raft_consensus.cc:3037] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.092443 19578 raft_consensus.cc:3037] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.093883 19578 raft_consensus.cc:2454] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 1.
I20260926 16:37:44.093876 19858 raft_consensus.cc:2454] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 1.
I20260926 16:37:44.094245 19651 leader_election.cc:302] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 25fc282343554c8bab9711ee90f0b074, 6405f2545dca406984b16732bfcb7ce3, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.094539 19938 raft_consensus.cc:2781] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:44.094823 19938 raft_consensus.cc:686] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 1 LEADER]: Becoming Leader. State: Replica: 6405f2545dca406984b16732bfcb7ce3, State: Running, Role: LEADER
I20260926 16:37:44.095207 19938 consensus_queue.cc:237] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.098582 19421 catalog_manager.cc:5495] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130). New cstate: current_term: 1 leader_uuid: "6405f2545dca406984b16732bfcb7ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } health_report { overall_health: UNKNOWN } } }
W20260926 16:37:44.129374 19905 tablet.cc:2369] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:44.140257 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.136305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.11.158.130:39367)
0926 16:37:44.136463 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.136467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.136580 (+ 113us) server_negotiation.cc:407] Connection header received
0926 16:37:44.136723 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.136725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.136782 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.136871 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.137152 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.137688 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.138549 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.138766 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.138930 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.139340 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.139828 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.139911 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.140054 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.140115 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 16:37:44.147184 19858 raft_consensus.cc:1261] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Refusing update from remote peer 6405f2545dca406984b16732bfcb7ce3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.147184 19578 raft_consensus.cc:1261] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Refusing update from remote peer 6405f2545dca406984b16732bfcb7ce3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.148175 19942 consensus_queue.cc:1035] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.148547 19938 consensus_queue.cc:1035] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:37:44.149825 19765 tablet.cc:2369] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:44.167042 19420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.167222 19420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.170024 19420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Sending DeleteTablet for 3 replicas of tablet 387d6dd75ce7497f9301cc02ea24ced4
I20260926 16:37:44.170969 19698 tablet_service.cc:1514] Processing DeleteTablet for tablet 387d6dd75ce7497f9301cc02ea24ced4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:37148
I20260926 16:37:44.171023 19558 tablet_service.cc:1514] Processing DeleteTablet for tablet 387d6dd75ce7497f9301cc02ea24ced4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60914
I20260926 16:37:44.171094 19821 tablet_service.cc:1514] Processing DeleteTablet for tablet 387d6dd75ce7497f9301cc02ea24ced4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41884
I20260926 16:37:44.171491 19955 tablet_replica.cc:330] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: stopping tablet replica
I20260926 16:37:44.171854 19955 raft_consensus.cc:2229] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:44.172322 19955 raft_consensus.cc:2258] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.172690 19956 tablet_replica.cc:330] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: stopping tablet replica
I20260926 16:37:44.172940 19420 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:44.173434 19420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:44.174001 19955 ts_tablet_manager.cc:1905] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.174507 19956 raft_consensus.cc:2229] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.174854 19956 raft_consensus.cc:2258] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.177304 19955 ts_tablet_manager.cc:1918] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:44.177425 19955 log.cc:1196] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/387d6dd75ce7497f9301cc02ea24ced4
I20260926 16:37:44.177624 19957 tablet_replica.cc:330] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: stopping tablet replica
I20260926 16:37:44.177791 19955 ts_tablet_manager.cc:1939] T 387d6dd75ce7497f9301cc02ea24ced4 P 6405f2545dca406984b16732bfcb7ce3: Deleting consensus metadata
I20260926 16:37:44.178001 19957 raft_consensus.cc:2229] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.179340 19698 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5a361122c34894a316bc800a11a65c (DEFAULT_TABLE table=test-table [id=7e690ba81fc14494b1fe3a71019e0d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.179417 19821 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5a361122c34894a316bc800a11a65c (DEFAULT_TABLE table=test-table [id=7e690ba81fc14494b1fe3a71019e0d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.179590 19698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5a361122c34894a316bc800a11a65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.179694 19821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5a361122c34894a316bc800a11a65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.179770 19956 ts_tablet_manager.cc:1905] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.181954 19933 tablet_bootstrap.cc:492] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: Bootstrap starting.
I20260926 16:37:44.182415 19934 tablet_bootstrap.cc:492] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: Bootstrap starting.
I20260926 16:37:44.182807 19956 ts_tablet_manager.cc:1918] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:44.182917 19956 log.cc:1196] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/387d6dd75ce7497f9301cc02ea24ced4
I20260926 16:37:44.183115 19933 tablet_bootstrap.cc:654] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.183262 19956 ts_tablet_manager.cc:1939] T 387d6dd75ce7497f9301cc02ea24ced4 P bb73f4a07e614ebe8b460b2cf486c022: Deleting consensus metadata
I20260926 16:37:44.183506 19934 tablet_bootstrap.cc:654] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.183758 19405 catalog_manager.cc:4841] TS 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367): tablet 387d6dd75ce7497f9301cc02ea24ced4 (table test-table [id=a1862137fb134461af9ea5a9835622be]) successfully deleted
I20260926 16:37:44.178342 19957 raft_consensus.cc:2258] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.185029 19408 catalog_manager.cc:4841] TS bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449): tablet 387d6dd75ce7497f9301cc02ea24ced4 (table test-table [id=a1862137fb134461af9ea5a9835622be]) successfully deleted
I20260926 16:37:44.185029 19558 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5a361122c34894a316bc800a11a65c (DEFAULT_TABLE table=test-table [id=7e690ba81fc14494b1fe3a71019e0d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.185145 19934 tablet_bootstrap.cc:492] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: No bootstrap required, opened a new log
I20260926 16:37:44.185235 19934 ts_tablet_manager.cc:1397] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:44.185299 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5a361122c34894a316bc800a11a65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.185730 19934 raft_consensus.cc:348] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.185874 19934 raft_consensus.cc:374] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.185913 19934 raft_consensus.cc:729] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25fc282343554c8bab9711ee90f0b074, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.186036 19934 consensus_queue.cc:260] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.186272 19934 ts_tablet_manager.cc:1428] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:44.186497 19957 ts_tablet_manager.cc:1905] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.186514 19933 tablet_bootstrap.cc:492] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: No bootstrap required, opened a new log
I20260926 16:37:44.186605 19933 ts_tablet_manager.cc:1397] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 16:37:44.187115 19933 raft_consensus.cc:348] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.187222 19933 raft_consensus.cc:374] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.187251 19933 raft_consensus.cc:729] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6405f2545dca406984b16732bfcb7ce3, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.187381 19933 consensus_queue.cc:260] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.187660 19933 ts_tablet_manager.cc:1428] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 16:37:44.188208 19932 tablet_bootstrap.cc:492] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: Bootstrap starting.
I20260926 16:37:44.189738 19957 ts_tablet_manager.cc:1918] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:44.189859 19957 log.cc:1196] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/387d6dd75ce7497f9301cc02ea24ced4
I20260926 16:37:44.190145 19957 ts_tablet_manager.cc:1939] T 387d6dd75ce7497f9301cc02ea24ced4 P 25fc282343554c8bab9711ee90f0b074: Deleting consensus metadata
I20260926 16:37:44.190428 19932 tablet_bootstrap.cc:654] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.191552 19407 catalog_manager.cc:4841] TS 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005): tablet 387d6dd75ce7497f9301cc02ea24ced4 (table test-table [id=a1862137fb134461af9ea5a9835622be]) successfully deleted
I20260926 16:37:44.192535 19932 tablet_bootstrap.cc:492] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: No bootstrap required, opened a new log
I20260926 16:37:44.192646 19932 ts_tablet_manager.cc:1397] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 16:37:44.193117 19932 raft_consensus.cc:348] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.193240 19932 raft_consensus.cc:374] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.193272 19932 raft_consensus.cc:729] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.193396 19932 consensus_queue.cc:260] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.194123 19932 ts_tablet_manager.cc:1428] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:44.260886 19940 raft_consensus.cc:482] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:44.261059 19940 raft_consensus.cc:504] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.262319 19940 leader_election.cc:288] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.267520 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.262607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39367 (local address 127.11.158.129:59171)
0926 16:37:44.262987 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.263006 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.263138 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.263458 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.263461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.263474 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:44.263636 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.263645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.264809 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.264812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.266026 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.266040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.266979 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.266984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.267122 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.267176 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.267259 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.267338 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":59,"thread_start_us":75,"threads_started":1}
I20260926 16:37:44.267515 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.262709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:59171 (local address 127.11.158.130:39367)
0926 16:37:44.262841 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.262846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.263107 (+ 261us) server_negotiation.cc:407] Connection header received
0926 16:37:44.263276 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.263279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.263331 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.263425 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:44.263780 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.264627 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.266213 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.266819 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.266945 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.267006 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.267298 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.267394 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:37:44.267618 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.262799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:53331 (local address 127.11.158.131:33005)
0926 16:37:44.262955 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.262959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.263259 (+ 300us) server_negotiation.cc:407] Connection header received
0926 16:37:44.263437 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.263441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.263490 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.263563 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:37:44.263868 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.264627 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.266382 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.266974 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.267081 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.267136 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.267423 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.267492 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:37:44.267952 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.262607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:33005 (local address 127.11.158.129:53331)
0926 16:37:44.263180 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.263194 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.263325 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.263585 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.263588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.263608 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:37:44.263740 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.263746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.265002 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.265007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.266243 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.266255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.267081 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.267085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.267279 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.267304 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.267737 (+ 433us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.267817 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":268,"thread_start_us":97,"threads_started":1}
I20260926 16:37:44.268384 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5a361122c34894a316bc800a11a65c" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6405f2545dca406984b16732bfcb7ce3" is_pre_election: true
I20260926 16:37:44.268597 19858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5a361122c34894a316bc800a11a65c" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074" is_pre_election: true
I20260926 16:37:44.268661 19718 raft_consensus.cc:2454] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 0.
I20260926 16:37:44.268739 19858 raft_consensus.cc:2454] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 0.
I20260926 16:37:44.269100 19509 leader_election.cc:302] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6405f2545dca406984b16732bfcb7ce3, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.269415 19940 raft_consensus.cc:2781] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:44.269479 19940 raft_consensus.cc:482] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:44.269526 19940 raft_consensus.cc:3037] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.270522 19940 raft_consensus.cc:504] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.270954 19940 leader_election.cc:288] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 election: Requested vote from peers 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.271325 19858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5a361122c34894a316bc800a11a65c" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074"
I20260926 16:37:44.271330 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5a361122c34894a316bc800a11a65c" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6405f2545dca406984b16732bfcb7ce3"
I20260926 16:37:44.271456 19718 raft_consensus.cc:3037] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.271464 19858 raft_consensus.cc:3037] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.272570 19858 raft_consensus.cc:2454] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 1.
I20260926 16:37:44.272845 19718 raft_consensus.cc:2454] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 1.
I20260926 16:37:44.272943 19511 leader_election.cc:302] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25fc282343554c8bab9711ee90f0b074, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.273115 19940 raft_consensus.cc:2781] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:44.273366 19940 raft_consensus.cc:686] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 1 LEADER]: Becoming Leader. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Running, Role: LEADER
I20260926 16:37:44.273631 19940 consensus_queue.cc:237] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.276093 19421 catalog_manager.cc:5495] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129). New cstate: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } health_report { overall_health: UNKNOWN } } }
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440663, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.190@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:44.319617 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.310259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34013 (local address 127.0.0.1:54602)
0926 16:37:44.310569 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.310586 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.310707 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.310977 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.310979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.311363 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:44.311458 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.311463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.312556 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.312558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.313209 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.313219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.313781 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.313788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.313890 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.314522 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:44.314534 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:44.316217 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:44.318116 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:44.318125 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:44.318130 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:44.318365 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:44.318638 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:44.318640 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:44.318642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:44.318719 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:44.319147 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:44.319153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:44.319268 (+ 115us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.319358 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.319476 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":127,"threads_started":1}
I20260926 16:37:44.319732 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.310375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54602 (local address 127.11.158.190:34013)
0926 16:37:44.310528 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.310534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.310669 (+ 135us) server_negotiation.cc:407] Connection header received
0926 16:37:44.310796 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.310799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.310848 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.310945 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:44.311577 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.312453 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.313310 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.313629 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.313753 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.316386 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:44.316390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:44.316393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:44.316420 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:44.318007 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:44.318460 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:44.318464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:44.318465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:44.318552 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:44.318801 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:44.318805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:44.318806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:44.319030 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:44.319145 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.319436 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.319551 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:37:44.323742 19421 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:54602:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 16:37:44.324573 19421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:44.329664 19698 tablet_service.cc:1467] Processing CreateTablet for tablet 46d4abc57de34992be318c7884580c58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e8e8aae3d6854d0e8e476a3f74129518]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:44.329666 19558 tablet_service.cc:1467] Processing CreateTablet for tablet 46d4abc57de34992be318c7884580c58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e8e8aae3d6854d0e8e476a3f74129518]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:44.329706 19821 tablet_service.cc:1467] Processing CreateTablet for tablet 46d4abc57de34992be318c7884580c58 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e8e8aae3d6854d0e8e476a3f74129518]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:37:44.330062 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d4abc57de34992be318c7884580c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.330062 19821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d4abc57de34992be318c7884580c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.330061 19698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d4abc57de34992be318c7884580c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.333029 19934 tablet_bootstrap.cc:492] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: Bootstrap starting.
I20260926 16:37:44.333447 19933 tablet_bootstrap.cc:492] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: Bootstrap starting.
I20260926 16:37:44.333464 19932 tablet_bootstrap.cc:492] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: Bootstrap starting.
I20260926 16:37:44.334074 19934 tablet_bootstrap.cc:654] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.334272 19933 tablet_bootstrap.cc:654] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.334456 19932 tablet_bootstrap.cc:654] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.335887 19932 tablet_bootstrap.cc:492] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: No bootstrap required, opened a new log
I20260926 16:37:44.335994 19932 ts_tablet_manager.cc:1397] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:44.336452 19932 raft_consensus.cc:348] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.336577 19932 raft_consensus.cc:374] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.336618 19932 raft_consensus.cc:729] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.336742 19932 consensus_queue.cc:260] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.336973 19940 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.337077 19940 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:44.337278 19934 tablet_bootstrap.cc:492] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: No bootstrap required, opened a new log
I20260926 16:37:44.337364 19934 ts_tablet_manager.cc:1397] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 16:37:44.337744 19932 ts_tablet_manager.cc:1428] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:44.337749 19934 raft_consensus.cc:348] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.337857 19934 raft_consensus.cc:374] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.337883 19934 raft_consensus.cc:729] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25fc282343554c8bab9711ee90f0b074, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.337993 19934 consensus_queue.cc:260] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.338222 19934 ts_tablet_manager.cc:1428] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:37:44.338245 19939 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.338435 19939 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:44.338740 19933 tablet_bootstrap.cc:492] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: No bootstrap required, opened a new log
I20260926 16:37:44.338821 19933 ts_tablet_manager.cc:1397] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260926 16:37:44.339200 19933 raft_consensus.cc:348] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.339310 19933 raft_consensus.cc:374] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.339342 19933 raft_consensus.cc:729] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6405f2545dca406984b16732bfcb7ce3, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.339466 19933 consensus_queue.cc:260] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.339743 19933 ts_tablet_manager.cc:1428] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:44.339923 19942 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.340049 19942 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:44.355185 19940 raft_consensus.cc:482] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:44.355324 19940 raft_consensus.cc:504] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.355811 19940 leader_election.cc:288] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.356374 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d4abc57de34992be318c7884580c58" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6405f2545dca406984b16732bfcb7ce3" is_pre_election: true
I20260926 16:37:44.356379 19858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d4abc57de34992be318c7884580c58" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074" is_pre_election: true
I20260926 16:37:44.356537 19858 raft_consensus.cc:2454] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 0.
I20260926 16:37:44.356537 19718 raft_consensus.cc:2454] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 0.
I20260926 16:37:44.356830 19511 leader_election.cc:302] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25fc282343554c8bab9711ee90f0b074, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.357052 19940 raft_consensus.cc:2781] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:44.357125 19940 raft_consensus.cc:482] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:44.357167 19940 raft_consensus.cc:3037] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.358232 19940 raft_consensus.cc:504] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.358636 19940 leader_election.cc:288] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 election: Requested vote from peers 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.358920 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d4abc57de34992be318c7884580c58" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6405f2545dca406984b16732bfcb7ce3"
I20260926 16:37:44.359063 19718 raft_consensus.cc:3037] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.359028 19858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d4abc57de34992be318c7884580c58" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074"
I20260926 16:37:44.359192 19858 raft_consensus.cc:3037] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.360090 19718 raft_consensus.cc:2454] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 1.
I20260926 16:37:44.360095 19858 raft_consensus.cc:2454] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 1.
I20260926 16:37:44.360419 19509 leader_election.cc:302] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6405f2545dca406984b16732bfcb7ce3, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.360591 19940 raft_consensus.cc:2781] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:44.360706 19940 raft_consensus.cc:686] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 LEADER]: Becoming Leader. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Running, Role: LEADER
I20260926 16:37:44.360848 19940 consensus_queue.cc:237] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.361509 19963 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: TxnStatusTablet state changed. Reason: New leader bb73f4a07e614ebe8b460b2cf486c022. Latest consensus state: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.361661 19963 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:44.361958 19975 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 16:37:44.362123 19975 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 16:37:44.362604 19419 catalog_manager.cc:5495] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129). New cstate: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:44.374514 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.370702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.11.158.129:36449)
0926 16:37:44.370822 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.370826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.370956 (+ 130us) server_negotiation.cc:407] Connection header received
0926 16:37:44.371134 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.371136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.371188 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.371296 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.371568 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.372118 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.373084 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.373301 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.373413 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.373710 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.374112 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.374233 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.374310 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.374366 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:37:44.374513 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.370624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36449 (local address 127.0.0.1:60926)
0926 16:37:44.370878 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.370894 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.371007 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.371340 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.371343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.371351 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:44.371461 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.371466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.372237 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.372242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.372951 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.372960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.373412 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.373415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.373506 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.373560 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.374217 (+ 657us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.374232 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.374300 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.374391 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260926 16:37:44.378006 19858 raft_consensus.cc:1261] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Refusing update from remote peer bb73f4a07e614ebe8b460b2cf486c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.378211 19718 raft_consensus.cc:1261] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Refusing update from remote peer bb73f4a07e614ebe8b460b2cf486c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.378473 19940 consensus_queue.cc:1035] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.378726 19963 consensus_queue.cc:1035] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.380255 19939 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: TxnStatusTablet state changed. Reason: New leader bb73f4a07e614ebe8b460b2cf486c022. Latest consensus state: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.380403 19939 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:44.381247 19942 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: TxnStatusTablet state changed. Reason: New leader bb73f4a07e614ebe8b460b2cf486c022. Latest consensus state: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.381356 19942 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:44.383157 19963 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.383283 19963 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:44.383381 19946 mvcc.cc:205] Tried to move back new op lower bound from 7333644961287049216 to 7333644961223282688. Current Snapshot: MvccSnapshot[applied={T|T < 7333644961287049216}]
I20260926 16:37:44.383692 19945 mvcc.cc:205] Tried to move back new op lower bound from 7333644961287049216 to 7333644961223282688. Current Snapshot: MvccSnapshot[applied={T|T < 7333644961287049216}]
I20260926 16:37:44.383884 19942 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.384007 19942 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P 6405f2545dca406984b16732bfcb7ce3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:44.387436 19963 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.387553 19963 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P bb73f4a07e614ebe8b460b2cf486c022: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:37:44.388307 19947 mvcc.cc:205] Tried to move back new op lower bound from 7333644961287049216 to 7333644961223282688. Current Snapshot: MvccSnapshot[applied={T|T < 7333644961287049216}]
I20260926 16:37:44.388464 19939 tablet_replica.cc:439] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } } }
I20260926 16:37:44.388597 19939 tablet_replica.cc:442] T 46d4abc57de34992be318c7884580c58 P 25fc282343554c8bab9711ee90f0b074: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:37:44.393934 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.389802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60938 (local address 127.11.158.129:36449)
0926 16:37:44.390339 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.390344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.390357 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:44.390399 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.390402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.390440 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.390531 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.390849 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.391413 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.392632 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.392993 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.393108 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.393363 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.393445 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.393517 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.393731 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.393794 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467}
I20260926 16:37:44.404879 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.401168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:46993 (local address 127.11.158.129:36449)
0926 16:37:44.401287 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.401292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.401499 (+ 207us) server_negotiation.cc:407] Connection header received
0926 16:37:44.401593 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.401596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.401662 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.401740 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.401956 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.402717 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.403578 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.403914 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.404028 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.404325 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.404428 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.404501 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.404662 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.404714 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:37:44.404889 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.401125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36449 (local address 127.11.158.129:46993)
0926 16:37:44.401413 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.401434 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.401521 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.401754 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.401756 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.401761 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:44.401863 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.401867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.402831 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.402833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.403464 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.403473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.404056 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.404059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.404143 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.404190 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.404540 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.404547 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.404633 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.404704 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":110,"threads_started":1}
I20260926 16:37:44.412312 19858 raft_consensus.cc:1261] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Refusing update from remote peer bb73f4a07e614ebe8b460b2cf486c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.412405 19718 raft_consensus.cc:1261] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Refusing update from remote peer bb73f4a07e614ebe8b460b2cf486c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.412741 19963 consensus_queue.cc:1035] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.412901 19940 consensus_queue.cc:1035] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.458205 19419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.458376 19419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.460395 19419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Sending DeleteTablet for 3 replicas of tablet 4a5a361122c34894a316bc800a11a65c
I20260926 16:37:44.460940 19698 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5a361122c34894a316bc800a11a65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:37148
I20260926 16:37:44.461090 19821 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5a361122c34894a316bc800a11a65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41884
I20260926 16:37:44.461151 19955 tablet_replica.cc:330] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: stopping tablet replica
I20260926 16:37:44.461263 19955 raft_consensus.cc:2229] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.461269 19957 tablet_replica.cc:330] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: stopping tablet replica
I20260926 16:37:44.461375 19955 raft_consensus.cc:2258] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.461414 19957 raft_consensus.cc:2229] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.461542 19957 raft_consensus.cc:2258] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.461738 19558 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5a361122c34894a316bc800a11a65c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60914
I20260926 16:37:44.461927 19956 tablet_replica.cc:330] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: stopping tablet replica
I20260926 16:37:44.462047 19956 raft_consensus.cc:2229] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:44.462321 19956 raft_consensus.cc:2258] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.462522 19957 ts_tablet_manager.cc:1905] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.463011 19956 ts_tablet_manager.cc:1905] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.465549 19956 ts_tablet_manager.cc:1918] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:44.465643 19956 log.cc:1196] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/4a5a361122c34894a316bc800a11a65c
I20260926 16:37:44.465904 19956 ts_tablet_manager.cc:1939] T 4a5a361122c34894a316bc800a11a65c P bb73f4a07e614ebe8b460b2cf486c022: Deleting consensus metadata
I20260926 16:37:44.466312 19955 ts_tablet_manager.cc:1905] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.466611 19408 catalog_manager.cc:4841] TS bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449): tablet 4a5a361122c34894a316bc800a11a65c (table test-table [id=7e690ba81fc14494b1fe3a71019e0d89]) successfully deleted
I20260926 16:37:44.467201 19957 ts_tablet_manager.cc:1918] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:44.467285 19957 log.cc:1196] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/4a5a361122c34894a316bc800a11a65c
I20260926 16:37:44.467527 19957 ts_tablet_manager.cc:1939] T 4a5a361122c34894a316bc800a11a65c P 25fc282343554c8bab9711ee90f0b074: Deleting consensus metadata
I20260926 16:37:44.468241 19407 catalog_manager.cc:4841] TS 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005): tablet 4a5a361122c34894a316bc800a11a65c (table test-table [id=7e690ba81fc14494b1fe3a71019e0d89]) successfully deleted
I20260926 16:37:44.469035 19955 ts_tablet_manager.cc:1918] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:44.469131 19955 log.cc:1196] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/4a5a361122c34894a316bc800a11a65c
I20260926 16:37:44.469364 19955 ts_tablet_manager.cc:1939] T 4a5a361122c34894a316bc800a11a65c P 6405f2545dca406984b16732bfcb7ce3: Deleting consensus metadata
I20260926 16:37:44.470077 19405 catalog_manager.cc:4841] TS 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367): tablet 4a5a361122c34894a316bc800a11a65c (table test-table [id=7e690ba81fc14494b1fe3a71019e0d89]) successfully deleted
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[19376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:37:44.493221 19470 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:44.485680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.11.158.190:34013)
0926 16:37:44.485846 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.485851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.486134 (+ 283us) server_negotiation.cc:407] Connection header received
0926 16:37:44.486252 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.486255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.486313 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.486415 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:44.487179 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.488013 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.489203 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.489515 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.489636 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.493090 (+ 3454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54614: BlockingRecv error: recv got EOF from 127.0.0.1:54614 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[19376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM
I20260926 16:37:44.504719 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.494958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.11.158.190:34013)
0926 16:37:44.495104 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.495109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.495243 (+ 134us) server_negotiation.cc:407] Connection header received
0926 16:37:44.495384 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.495386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.495430 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.495505 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:44.496303 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.496952 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.498062 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.498278 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.498367 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.501234 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:44.501238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:44.501241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:44.501271 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:44.502893 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:44.503316 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:44.503319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:44.503321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:44.503413 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:44.503726 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:44.503729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:44.503731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:44.503961 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:44.504100 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.504401 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.504525 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:37:44.507627 19419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:37:44.508111 19419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:44.513087 19558 tablet_service.cc:1467] Processing CreateTablet for tablet e50b0005054745b999ff519447c7d293 (DEFAULT_TABLE table=test-table [id=28358d8604e544028549018d6999d9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.513201 19698 tablet_service.cc:1467] Processing CreateTablet for tablet e50b0005054745b999ff519447c7d293 (DEFAULT_TABLE table=test-table [id=28358d8604e544028549018d6999d9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.513406 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e50b0005054745b999ff519447c7d293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.513440 19698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e50b0005054745b999ff519447c7d293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.515872 19933 tablet_bootstrap.cc:492] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: Bootstrap starting.
I20260926 16:37:44.516058 19932 tablet_bootstrap.cc:492] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Bootstrap starting.
I20260926 16:37:44.517042 19933 tablet_bootstrap.cc:654] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.517114 19932 tablet_bootstrap.cc:654] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.517966 19821 tablet_service.cc:1467] Processing CreateTablet for tablet e50b0005054745b999ff519447c7d293 (DEFAULT_TABLE table=test-table [id=28358d8604e544028549018d6999d9c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.518299 19821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e50b0005054745b999ff519447c7d293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.518464 19932 tablet_bootstrap.cc:492] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: No bootstrap required, opened a new log
I20260926 16:37:44.518549 19932 ts_tablet_manager.cc:1397] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:37:44.519001 19932 raft_consensus.cc:348] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.519121 19932 raft_consensus.cc:374] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.519171 19932 raft_consensus.cc:729] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.519304 19932 consensus_queue.cc:260] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.519577 19932 ts_tablet_manager.cc:1428] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:37:44.521153 19934 tablet_bootstrap.cc:492] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: Bootstrap starting.
I20260926 16:37:44.522118 19934 tablet_bootstrap.cc:654] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.523391 19934 tablet_bootstrap.cc:492] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: No bootstrap required, opened a new log
I20260926 16:37:44.523478 19934 ts_tablet_manager.cc:1397] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:44.523912 19934 raft_consensus.cc:348] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.524035 19934 raft_consensus.cc:374] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.524065 19934 raft_consensus.cc:729] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25fc282343554c8bab9711ee90f0b074, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.524204 19934 consensus_queue.cc:260] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.524540 19934 ts_tablet_manager.cc:1428] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:44.525342 19933 tablet_bootstrap.cc:492] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: No bootstrap required, opened a new log
I20260926 16:37:44.525466 19933 ts_tablet_manager.cc:1397] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260926 16:37:44.526017 19933 raft_consensus.cc:348] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.526167 19933 raft_consensus.cc:374] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.526211 19933 raft_consensus.cc:729] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6405f2545dca406984b16732bfcb7ce3, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.526366 19933 consensus_queue.cc:260] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.526670 19933 ts_tablet_manager.cc:1428] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 16:37:44.668212 19963 raft_consensus.cc:482] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:44.668375 19963 raft_consensus.cc:504] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.668821 19963 leader_election.cc:288] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.669238 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50b0005054745b999ff519447c7d293" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6405f2545dca406984b16732bfcb7ce3" is_pre_election: true
I20260926 16:37:44.669399 19718 raft_consensus.cc:2454] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 0.
I20260926 16:37:44.669499 19857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50b0005054745b999ff519447c7d293" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074" is_pre_election: true
I20260926 16:37:44.669673 19857 raft_consensus.cc:2454] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 0.
I20260926 16:37:44.669742 19509 leader_election.cc:302] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6405f2545dca406984b16732bfcb7ce3, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.669987 19963 raft_consensus.cc:2781] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:44.670079 19963 raft_consensus.cc:482] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:44.670132 19963 raft_consensus.cc:3037] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.671105 19963 raft_consensus.cc:504] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.671490 19963 leader_election.cc:288] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 election: Requested vote from peers 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.671901 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50b0005054745b999ff519447c7d293" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6405f2545dca406984b16732bfcb7ce3"
I20260926 16:37:44.671890 19857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e50b0005054745b999ff519447c7d293" candidate_uuid: "bb73f4a07e614ebe8b460b2cf486c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074"
I20260926 16:37:44.672030 19718 raft_consensus.cc:3037] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.672046 19857 raft_consensus.cc:3037] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Advancing to term 1
W20260926 16:37:44.673015 19625 tablet.cc:2369] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:37:44.673118 19857 raft_consensus.cc:2454] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 1.
I20260926 16:37:44.673352 19718 raft_consensus.cc:2454] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb73f4a07e614ebe8b460b2cf486c022 in term 1.
I20260926 16:37:44.673514 19511 leader_election.cc:302] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25fc282343554c8bab9711ee90f0b074, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.673758 19963 raft_consensus.cc:2781] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:44.673858 19963 raft_consensus.cc:686] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 LEADER]: Becoming Leader. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Running, Role: LEADER
I20260926 16:37:44.673995 19963 consensus_queue.cc:237] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.675845 19419 catalog_manager.cc:5495] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129). New cstate: current_term: 1 leader_uuid: "bb73f4a07e614ebe8b460b2cf486c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:44.712658 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.709491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.11.158.129:36449)
0926 16:37:44.709637 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.709641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.709696 (+ 55us) server_negotiation.cc:407] Connection header received
0926 16:37:44.709828 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.709831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.709873 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.709943 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.710212 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.710713 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.711516 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.711725 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.711803 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.712170 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.712259 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.712386 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.712505 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.712549 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:37:44.714654 19718 raft_consensus.cc:1261] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Refusing update from remote peer bb73f4a07e614ebe8b460b2cf486c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.714687 19857 raft_consensus.cc:1261] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Refusing update from remote peer bb73f4a07e614ebe8b460b2cf486c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.715059 19980 consensus_queue.cc:1035] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.715248 19963 consensus_queue.cc:1035] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.722167 19419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.722349 19419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.724539 19419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Sending DeleteTablet for 3 replicas of tablet e50b0005054745b999ff519447c7d293
I20260926 16:37:44.725214 19698 tablet_service.cc:1514] Processing DeleteTablet for tablet e50b0005054745b999ff519447c7d293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:37148
I20260926 16:37:44.725226 19558 tablet_service.cc:1514] Processing DeleteTablet for tablet e50b0005054745b999ff519447c7d293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60914
I20260926 16:37:44.725250 19821 tablet_service.cc:1514] Processing DeleteTablet for tablet e50b0005054745b999ff519447c7d293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41884
I20260926 16:37:44.725400 19955 tablet_replica.cc:330] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: stopping tablet replica
I20260926 16:37:44.725418 19956 tablet_replica.cc:330] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: stopping tablet replica
I20260926 16:37:44.725513 19957 tablet_replica.cc:330] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: stopping tablet replica
I20260926 16:37:44.725522 19956 raft_consensus.cc:2229] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:44.725520 19955 raft_consensus.cc:2229] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.725663 19955 raft_consensus.cc:2258] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.725688 19957 raft_consensus.cc:2229] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.725733 19956 raft_consensus.cc:2258] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.725849 19957 raft_consensus.cc:2258] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.726557 19956 ts_tablet_manager.cc:1905] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.726586 19955 ts_tablet_manager.cc:1905] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.729472 19956 ts_tablet_manager.cc:1918] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:44.729470 19955 ts_tablet_manager.cc:1918] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:44.729583 19956 log.cc:1196] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/e50b0005054745b999ff519447c7d293
I20260926 16:37:44.729583 19955 log.cc:1196] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/e50b0005054745b999ff519447c7d293
I20260926 16:37:44.729483 19419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 16:37:44.729916 19956 ts_tablet_manager.cc:1939] T e50b0005054745b999ff519447c7d293 P bb73f4a07e614ebe8b460b2cf486c022: Deleting consensus metadata
I20260926 16:37:44.729916 19955 ts_tablet_manager.cc:1939] T e50b0005054745b999ff519447c7d293 P 6405f2545dca406984b16732bfcb7ce3: Deleting consensus metadata
W20260926 16:37:44.729997 19419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:37:44.730192 19821 tablet_service.cc:1514] Processing DeleteTablet for tablet e50b0005054745b999ff519447c7d293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41884
I20260926 16:37:44.730926 19405 catalog_manager.cc:4841] TS 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367): tablet e50b0005054745b999ff519447c7d293 (table test-table [id=28358d8604e544028549018d6999d9c5]) successfully deleted
I20260926 16:37:44.731092 19408 catalog_manager.cc:4841] TS bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449): tablet e50b0005054745b999ff519447c7d293 (table test-table [id=28358d8604e544028549018d6999d9c5]) successfully deleted
I20260926 16:37:44.731552 19957 ts_tablet_manager.cc:1905] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.734615 19957 ts_tablet_manager.cc:1918] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:37:44.734726 19957 log.cc:1196] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/e50b0005054745b999ff519447c7d293
I20260926 16:37:44.735044 19957 ts_tablet_manager.cc:1939] T e50b0005054745b999ff519447c7d293 P 25fc282343554c8bab9711ee90f0b074: Deleting consensus metadata
I20260926 16:37:44.735286 19558 tablet_service.cc:1467] Processing CreateTablet for tablet b7c0f5f49b234508a0918814d57d00ff (DEFAULT_TABLE table=test-table [id=963b27b6012942d69d0285c77fa8ccaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.735594 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c0f5f49b234508a0918814d57d00ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.735594 19821 tablet_service.cc:1467] Processing CreateTablet for tablet b7c0f5f49b234508a0918814d57d00ff (DEFAULT_TABLE table=test-table [id=963b27b6012942d69d0285c77fa8ccaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:37:44.735881 19407 catalog_manager.cc:4841] TS 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005): tablet e50b0005054745b999ff519447c7d293 (table test-table [id=28358d8604e544028549018d6999d9c5]) successfully deleted
I20260926 16:37:44.736052 19821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c0f5f49b234508a0918814d57d00ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.737787 19932 tablet_bootstrap.cc:492] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022: Bootstrap starting.
I20260926 16:37:44.738124 19698 tablet_service.cc:1467] Processing CreateTablet for tablet b7c0f5f49b234508a0918814d57d00ff (DEFAULT_TABLE table=test-table [id=963b27b6012942d69d0285c77fa8ccaa]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 16:37:44.738286 19407 catalog_manager.cc:4808] TS 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005): delete failed for tablet e50b0005054745b999ff519447c7d293 because the tablet was not found. No further retry: Not found: Tablet not found: e50b0005054745b999ff519447c7d293
I20260926 16:37:44.738389 19698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7c0f5f49b234508a0918814d57d00ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:44.738823 19932 tablet_bootstrap.cc:654] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.740070 19932 tablet_bootstrap.cc:492] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022: No bootstrap required, opened a new log
I20260926 16:37:44.740166 19932 ts_tablet_manager.cc:1397] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:44.740657 19933 tablet_bootstrap.cc:492] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: Bootstrap starting.
I20260926 16:37:44.740607 19932 raft_consensus.cc:348] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.740754 19934 tablet_bootstrap.cc:492] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: Bootstrap starting.
I20260926 16:37:44.740758 19932 raft_consensus.cc:374] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.740793 19932 raft_consensus.cc:729] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb73f4a07e614ebe8b460b2cf486c022, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.740932 19932 consensus_queue.cc:260] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.741299 19932 ts_tablet_manager.cc:1428] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:44.741514 19933 tablet_bootstrap.cc:654] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.741544 19934 tablet_bootstrap.cc:654] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:44.742703 19933 tablet_bootstrap.cc:492] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: No bootstrap required, opened a new log
I20260926 16:37:44.742787 19933 ts_tablet_manager.cc:1397] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:44.742834 19934 tablet_bootstrap.cc:492] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: No bootstrap required, opened a new log
I20260926 16:37:44.742897 19934 ts_tablet_manager.cc:1397] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:37:44.743193 19933 raft_consensus.cc:348] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.743306 19933 raft_consensus.cc:374] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.743341 19933 raft_consensus.cc:729] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6405f2545dca406984b16732bfcb7ce3, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.743294 19934 raft_consensus.cc:348] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.743403 19934 raft_consensus.cc:374] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:44.743434 19934 raft_consensus.cc:729] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25fc282343554c8bab9711ee90f0b074, State: Initialized, Role: FOLLOWER
I20260926 16:37:44.743461 19933 consensus_queue.cc:260] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.743546 19934 consensus_queue.cc:260] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.743731 19933 ts_tablet_manager.cc:1428] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:37:44.743794 19934 ts_tablet_manager.cc:1428] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:37:44.759267 19942 raft_consensus.cc:482] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:37:44.759425 19942 raft_consensus.cc:504] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.759840 19942 leader_election.cc:288] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.760303 19578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c0f5f49b234508a0918814d57d00ff" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb73f4a07e614ebe8b460b2cf486c022" is_pre_election: true
I20260926 16:37:44.760305 19857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c0f5f49b234508a0918814d57d00ff" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074" is_pre_election: true
I20260926 16:37:44.760520 19578 raft_consensus.cc:2454] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 0.
I20260926 16:37:44.760529 19857 raft_consensus.cc:2454] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 0.
I20260926 16:37:44.760838 19652 leader_election.cc:302] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6405f2545dca406984b16732bfcb7ce3, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.761031 19942 raft_consensus.cc:2781] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:37:44.761092 19942 raft_consensus.cc:482] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:37:44.761168 19942 raft_consensus.cc:3037] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.762298 19942 raft_consensus.cc:504] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.762638 19942 leader_election.cc:288] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 election: Requested vote from peers bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449), 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005)
I20260926 16:37:44.763003 19578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c0f5f49b234508a0918814d57d00ff" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb73f4a07e614ebe8b460b2cf486c022"
I20260926 16:37:44.763073 19857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7c0f5f49b234508a0918814d57d00ff" candidate_uuid: "6405f2545dca406984b16732bfcb7ce3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fc282343554c8bab9711ee90f0b074"
I20260926 16:37:44.763123 19578 raft_consensus.cc:3037] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.763224 19857 raft_consensus.cc:3037] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:44.764120 19578 raft_consensus.cc:2454] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 1.
I20260926 16:37:44.764166 19857 raft_consensus.cc:2454] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6405f2545dca406984b16732bfcb7ce3 in term 1.
I20260926 16:37:44.764437 19652 leader_election.cc:302] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6405f2545dca406984b16732bfcb7ce3, bb73f4a07e614ebe8b460b2cf486c022; no voters:
I20260926 16:37:44.764596 19942 raft_consensus.cc:2781] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:44.764751 19942 raft_consensus.cc:686] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 1 LEADER]: Becoming Leader. State: Replica: 6405f2545dca406984b16732bfcb7ce3, State: Running, Role: LEADER
I20260926 16:37:44.764925 19942 consensus_queue.cc:237] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } }
I20260926 16:37:44.766634 19419 catalog_manager.cc:5495] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130). New cstate: current_term: 1 leader_uuid: "6405f2545dca406984b16732bfcb7ce3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6405f2545dca406984b16732bfcb7ce3" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20260926 16:37:44.787325 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.783916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.11.158.130:39367)
0926 16:37:44.784142 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.784148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.784164 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:44.784283 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.784289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.784386 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.784496 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.784754 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.785305 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.786168 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.786412 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.786540 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.786794 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.786879 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.786968 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.787068 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.787121 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":70,"threads_started":1}
I20260926 16:37:44.797175 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.794207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:59613 (local address 127.11.158.129:36449)
0926 16:37:44.794351 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.794355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.794450 (+ 95us) server_negotiation.cc:407] Connection header received
0926 16:37:44.794562 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.794564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.794660 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.794740 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.794951 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.795437 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.796235 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.796469 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.796574 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.796814 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.796877 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.796932 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.797027 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.797066 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 16:37:44.797191 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.794052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36449 (local address 127.11.158.130:59613)
0926 16:37:44.794342 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.794355 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.794442 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.794760 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.794762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.794769 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:44.794852 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.794856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.795523 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.795525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.796126 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.796133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.796587 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.796589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.796650 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.796695 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.796949 (+ 254us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.796954 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.797001 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.797064 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260926 16:37:44.802775 19857 raft_consensus.cc:1261] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Refusing update from remote peer 6405f2545dca406984b16732bfcb7ce3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.802932 19578 raft_consensus.cc:1261] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Refusing update from remote peer 6405f2545dca406984b16732bfcb7ce3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:37:44.803225 20009 consensus_queue.cc:1035] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25fc282343554c8bab9711ee90f0b074" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.803364 19942 consensus_queue.cc:1035] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb73f4a07e614ebe8b460b2cf486c022" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:37:44.822319 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.818908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39367 (local address 127.11.158.129:46699)
0926 16:37:44.819036 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.819055 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:44.819146 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:44.819457 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:44.819460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:44.819472 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:37:44.819598 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.819604 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.820276 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.820279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.821015 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:44.821024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.821539 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.821545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:44.821633 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.821684 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.822001 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.822008 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:37:44.822097 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:37:44.822185 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 16:37:44.822319 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.819054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:46699 (local address 127.11.158.130:39367)
0926 16:37:44.819193 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:44.819200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:44.819216 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:44.819260 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:44.819263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:44.819316 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:44.819395 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:37:44.819681 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.820167 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.821140 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:44.821408 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:44.821518 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:44.821819 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:37:44.821906 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:37:44.821982 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:44.822112 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 16:37:44.822167 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:37:44.844169 19419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.844326 19419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:37:44.846361 19419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 472d537977394f2ab338d92a970a3cf3: Sending DeleteTablet for 3 replicas of tablet b7c0f5f49b234508a0918814d57d00ff
I20260926 16:37:44.846916 19558 tablet_service.cc:1514] Processing DeleteTablet for tablet b7c0f5f49b234508a0918814d57d00ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60914
I20260926 16:37:44.846916 19698 tablet_service.cc:1514] Processing DeleteTablet for tablet b7c0f5f49b234508a0918814d57d00ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:37148
I20260926 16:37:44.847043 19821 tablet_service.cc:1514] Processing DeleteTablet for tablet b7c0f5f49b234508a0918814d57d00ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:37:44 UTC) from {username='oryx', principal='oryx/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41884
I20260926 16:37:44.847113 19956 tablet_replica.cc:330] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022: stopping tablet replica
I20260926 16:37:44.847131 19955 tablet_replica.cc:330] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: stopping tablet replica
I20260926 16:37:44.847229 19957 tablet_replica.cc:330] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: stopping tablet replica
I20260926 16:37:44.847231 19955 raft_consensus.cc:2229] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:37:44.847231 19956 raft_consensus.cc:2229] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.847338 19956 raft_consensus.cc:2258] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.847365 19957 raft_consensus.cc:2229] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:37:44.847424 19955 raft_consensus.cc:2258] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.847499 19957 raft_consensus.cc:2258] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:37:44.848102 19956 ts_tablet_manager.cc:1905] T b7c0f5f49b234508a0918814d57d00ff P bb73f4a07e614ebe8b460b2cf486c022: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.848101 19957 ts_tablet_manager.cc:1905] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.848146 19955 ts_tablet_manager.cc:1905] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:37:44.848454 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 19493
I20260926 16:37:44.850534 19957 ts_tablet_manager.cc:1918] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:44.850636 19957 log.cc:1196] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/b7c0f5f49b234508a0918814d57d00ff
I20260926 16:37:44.850888 19957 ts_tablet_manager.cc:1939] T b7c0f5f49b234508a0918814d57d00ff P 25fc282343554c8bab9711ee90f0b074: Deleting consensus metadata
I20260926 16:37:44.851329 19955 ts_tablet_manager.cc:1918] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:37:44.851457 19955 log.cc:1196] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/b7c0f5f49b234508a0918814d57d00ff
I20260926 16:37:44.851733 19955 ts_tablet_manager.cc:1939] T b7c0f5f49b234508a0918814d57d00ff P 6405f2545dca406984b16732bfcb7ce3: Deleting consensus metadata
I20260926 16:37:44.852232 19407 catalog_manager.cc:4841] TS 25fc282343554c8bab9711ee90f0b074 (127.11.158.131:33005): tablet b7c0f5f49b234508a0918814d57d00ff (table test-table [id=963b27b6012942d69d0285c77fa8ccaa]) successfully deleted
I20260926 16:37:44.852509 19405 catalog_manager.cc:4841] TS 6405f2545dca406984b16732bfcb7ce3 (127.11.158.130:39367): tablet b7c0f5f49b234508a0918814d57d00ff (table test-table [id=963b27b6012942d69d0285c77fa8ccaa]) successfully deleted
W20260926 16:37:44.857959 19408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:36449 (error 108)
I20260926 16:37:44.858692 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 19633
I20260926 16:37:44.858964 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:44.858552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36449 (local address 127.0.0.1:60970)
0926 16:37:44.858749 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:44.858843 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:36449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":53,"threads_started":1}
W20260926 16:37:44.860005 19408 catalog_manager.cc:4584] TS bb73f4a07e614ebe8b460b2cf486c022 (127.11.158.129:36449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7c0f5f49b234508a0918814d57d00ff: Network error: Client connection negotiation failed: client connection to 127.11.158.129:36449: connect: Connection refused (error 111)
I20260926 16:37:44.865367 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 19773
I20260926 16:37:44.871179 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 19389
2026-09-26T16:37:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3138 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[20022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[20022](info): set up 2 sockets
Sep 26 16:37:44 dist-test-slave-p39x krb5kdc[20022](info): commencing operation
krb5kdc: starting...
W20260926 16:37:45.913774 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:45 dist-test-slave-p39x krb5kdc[20022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:46.040238 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:46053
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:39725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:46053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:46.157351 20035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:46.157624 20035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:46.157665 20035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:46.161578 20035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:46.161633 20035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:46.161651 20035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:46.161669 20035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:46.161684 20035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:46.166882 20035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:39725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:46053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:46053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:46.167896 20035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:46.169029 20035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:46.177336 20040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:46.177336 20043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:46.177350 20041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:46.177603 20035 server_base.cc:1034] running on GCE node
I20260926 16:37:46.178310 20035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:46.179428 20035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:46.180598 20035 hybrid_clock.cc:648] HybridClock initialized: now 1790440666180585 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T16:37:46Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1316 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:37:46 dist-test-slave-p39x krb5kdc[20048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:37:46 dist-test-slave-p39x krb5kdc[20048](info): set up 2 sockets
Sep 26 16:37:46 dist-test-slave-p39x krb5kdc[20048](info): commencing operation
krb5kdc: starting...
W20260926 16:37:47.266073 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:37:47 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:37:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:37:47.391407 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39029
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:39029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:47.506301 20061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:47.506661 20061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:47.506739 20061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:47.511482 20061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:47.511638 20061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:47.511683 20061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:47.511754 20061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:47.511787 20061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:47.517971 20061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:39029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:47.519215 20061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:47.520326 20061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:47.529947 20067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:47.529963 20066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:47.530242 20069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:47.530216 20061 server_base.cc:1034] running on GCE node
I20260926 16:37:47.530905 20061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:47.532152 20061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:47.533356 20061 hybrid_clock.cc:648] HybridClock initialized: now 1790440667533334 us; error 39 us; skew 500 ppm
Sep 26 16:37:47 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440667, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:47.543373 20061 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:47.544529 20061 webserver.cc:460] Webserver started at http://127.11.158.190:33163/ using document root <none> and password file <none>
I20260926 16:37:47.545107 20061 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:47.545171 20061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:47.545349 20061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:47.547617 20061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "26c51ff5b7364414bd6fcf32445255be"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "db7b334c2849cd98aec71409e738c841"
server_key_iv: "8baf34d2026b302cad9cfba325cef8b8"
server_key_version: "encryptionkey@0"
I20260926 16:37:47.548195 20061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "26c51ff5b7364414bd6fcf32445255be"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "db7b334c2849cd98aec71409e738c841"
server_key_iv: "8baf34d2026b302cad9cfba325cef8b8"
server_key_version: "encryptionkey@0"
I20260926 16:37:47.552292 20061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260926 16:37:47.555217 20076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:47.556867 20061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:47.557020 20061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "26c51ff5b7364414bd6fcf32445255be"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "db7b334c2849cd98aec71409e738c841"
server_key_iv: "8baf34d2026b302cad9cfba325cef8b8"
server_key_version: "encryptionkey@0"
I20260926 16:37:47.557155 20061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:47.579782 20061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:47.583463 20061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:47.583709 20061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:47.591385 20061 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:39029
I20260926 16:37:47.591471 20128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:39029 every 8 connection(s)
I20260926 16:37:47.592546 20061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:47.595479 20129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:37:47.598503 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20061
I20260926 16:37:47.598618 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:37:47.598924 11898 external_mini_cluster.cc:1428] Setting key f15119660263e7b284ed3e23cd12e26b
I20260926 16:37:47.601954 20129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Bootstrap starting.
I20260926 16:37:47.604149 20129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Neither blocks nor log segments found. Creating new log.
I20260926 16:37:47.604957 20129 log.cc:824] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:47.607151 20129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: No bootstrap required, opened a new log
Sep 26 16:37:47 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:47.611068 20129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:47.611312 20129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:37:47.611374 20129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c51ff5b7364414bd6fcf32445255be, State: Initialized, Role: FOLLOWER
I20260926 16:37:47.611871 20129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:47.612042 20129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:47.612125 20129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:47.612252 20129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:37:47.613395 20129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:47.613788 20129 leader_election.cc:302] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c51ff5b7364414bd6fcf32445255be; no voters:
I20260926 16:37:47.614184 20129 leader_election.cc:288] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:37:47.614267 20134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:37:47.614478 20134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 LEADER]: Becoming Leader. State: Replica: 26c51ff5b7364414bd6fcf32445255be, State: Running, Role: LEADER
I20260926 16:37:47.614779 20134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:47.615221 20132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:47.600518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.11.158.190:39029)
0926 16:37:47.600988 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:47.601007 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:47.601042 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:37:47.601666 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:47.601684 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:47.602010 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:47.602388 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:47.603163 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:47.604151 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:47.605214 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:47.605959 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:47.606148 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:47.609194 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:47.609200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:47.609216 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:47.609246 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:47.611892 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:47.612480 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:47.612485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:47.612490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:47.612607 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:47.612965 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:47.612969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:47.612970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:47.613406 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:47.613652 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:47.613872 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 16:37:47.614226 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":164,"threads_started":1}
I20260926 16:37:47.615526 20129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:47.616555 20135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c51ff5b7364414bd6fcf32445255be. Latest consensus state: current_term: 1 leader_uuid: "26c51ff5b7364414bd6fcf32445255be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } } }
I20260926 16:37:47.616709 20135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:47.617030 20136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c51ff5b7364414bd6fcf32445255be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } } }
I20260926 16:37:47.617123 20136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:47.618188 20143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:47.621376 20143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:47.627408 20143 catalog_manager.cc:1347] Generated new cluster ID: 7e3866c74c994131832357c263028bff
I20260926 16:37:47.627485 20143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:47.642614 20143 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:37:47.644019 20143 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:47.654697 20143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Generated new TSK 0
I20260926 16:37:47.655580 20143 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:37:47.693624 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39029
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:47.804905 20155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:47.805188 20155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:47.805266 20155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:47.809312 20155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:47.809391 20155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:47.809482 20155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:47.814766 20155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:47.815896 20155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:47.817103 20155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:47.826854 20163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:47.826862 20160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:47.826915 20155 server_base.cc:1034] running on GCE node
W20260926 16:37:47.826862 20161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:47.827523 20155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:47.828168 20155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:47.829372 20155 hybrid_clock.cc:648] HybridClock initialized: now 1790440667829339 us; error 52 us; skew 500 ppm
Sep 26 16:37:47 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440667, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:47.832155 20155 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:37:47.833214 20155 webserver.cc:460] Webserver started at http://127.11.158.129:39725/ using document root <none> and password file <none>
I20260926 16:37:47.833801 20155 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:47.833899 20155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:47.834147 20155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:47.836863 20155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "4ac584f51bde4ed095deb9dd8faccabd"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "a6e8d7ff7ef8a3be1ef831d712c0813d"
server_key_iv: "a03b0f38ff0974dd0bc91a9a922ad491"
server_key_version: "encryptionkey@0"
I20260926 16:37:47.837487 20155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "4ac584f51bde4ed095deb9dd8faccabd"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "a6e8d7ff7ef8a3be1ef831d712c0813d"
server_key_iv: "a03b0f38ff0974dd0bc91a9a922ad491"
server_key_version: "encryptionkey@0"
I20260926 16:37:47.841380 20155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:37:47.843997 20170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:47.845302 20155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:37:47.845474 20155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "4ac584f51bde4ed095deb9dd8faccabd"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "a6e8d7ff7ef8a3be1ef831d712c0813d"
server_key_iv: "a03b0f38ff0974dd0bc91a9a922ad491"
server_key_version: "encryptionkey@0"
I20260926 16:37:47.845604 20155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:47.875361 20155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:47.878400 20155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:47.878646 20155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:47.879287 20155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:47.880193 20155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:47.880268 20155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:47.880383 20155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:47.880453 20155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:47.889966 20155 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:45067
I20260926 16:37:47.890007 20283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:45067 every 8 connection(s)
I20260926 16:37:47.890885 20155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
Sep 26 16:37:47 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440667, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:47.900652 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20155
I20260926 16:37:47.900795 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:37:47.901175 11898 external_mini_cluster.cc:1428] Setting key 8cc2fdd554d2899434d21bfd38eaab17
I20260926 16:37:47.904569 20132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:47.892723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:35079 (local address 127.11.158.190:39029)
0926 16:37:47.892858 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:47.892861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:47.893840 (+ 979us) server_negotiation.cc:407] Connection header received
0926 16:37:47.894680 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:47.894683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:47.894733 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:47.894815 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:47.895920 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:47.896593 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:47.897695 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:47.897917 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:47.898023 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:47.900553 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:47.900557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:47.900560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:47.900592 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:47.902295 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:47.902862 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:47.902867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:47.902870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:47.902971 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:47.903293 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:47.903300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:47.903304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:47.903584 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:47.903696 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:47.904144 (+ 448us) server_negotiation.cc:299] Negotiation successful
0926 16:37:47.904295 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:37:47.905421 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:47.893103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:35079)
0926 16:37:47.893722 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:47.893759 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:47.894469 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:47.894962 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:47.894968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:47.895321 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:47.895767 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:47.895780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:47.896714 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:47.896716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:47.897567 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:47.897576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:47.898065 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:47.898067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:47.898200 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:47.898806 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:47.898822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:47.900322 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:47.902409 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:47.902413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:47.902426 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:47.902764 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:47.903073 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:47.903075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:47.903077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:47.903198 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:47.903698 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:47.903702 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:47.903912 (+ 210us) client_negotiation.cc:769] Sending connection context
0926 16:37:47.904061 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 16:37:47.904316 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":113,"threads_started":1}
I20260926 16:37:47.906653 20284 heartbeater.cc:347] Connected to a master server at 127.11.158.190:39029
I20260926 16:37:47.906898 20284 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:47.907393 20284 heartbeater.cc:511] Master 127.11.158.190:39029 requested a full tablet report, sending...
I20260926 16:37:47.909016 20091 ts_manager.cc:194] Registered new tserver with Master: 4ac584f51bde4ed095deb9dd8faccabd (127.11.158.129:45067)
I20260926 16:37:47.910220 20091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:35079
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:37:47.929203 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39029
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:48.038203 20289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:48.038450 20289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:48.038491 20289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:48.042409 20289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:48.042461 20289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:48.042516 20289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:48.047567 20289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:48.048501 20289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:48.049537 20289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:48.058851 20297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.058854 20295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.058851 20294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:48.058903 20289 server_base.cc:1034] running on GCE node
I20260926 16:37:48.059500 20289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:48.060062 20289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:48.061225 20289 hybrid_clock.cc:648] HybridClock initialized: now 1790440668061179 us; error 58 us; skew 500 ppm
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:48.064118 20289 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:37:48.065021 20289 webserver.cc:460] Webserver started at http://127.11.158.130:34295/ using document root <none> and password file <none>
I20260926 16:37:48.065563 20289 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:48.065637 20289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:48.065894 20289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:48.068531 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "2c63e1ab6152438f84f138c3f4941c9b"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "2c5d99440baf7c80369f04bb65dba1cc"
server_key_iv: "481a64f6afdd36bccb4ae022cff3ce36"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.069203 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "2c63e1ab6152438f84f138c3f4941c9b"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "2c5d99440baf7c80369f04bb65dba1cc"
server_key_iv: "481a64f6afdd36bccb4ae022cff3ce36"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.072897 20289 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:37:48.075309 20304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.076510 20289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:37:48.076635 20289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "2c63e1ab6152438f84f138c3f4941c9b"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "2c5d99440baf7c80369f04bb65dba1cc"
server_key_iv: "481a64f6afdd36bccb4ae022cff3ce36"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.076745 20289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:48.092257 20289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:48.095427 20289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:48.095626 20289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:48.096202 20289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:48.097069 20289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:48.097139 20289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.097208 20289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:48.097261 20289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.105753 20289 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:37559
I20260926 16:37:48.105822 20417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:37559 every 8 connection(s)
I20260926 16:37:48.106693 20289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:48.115245 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20289
I20260926 16:37:48.115423 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:37:48.115782 11898 external_mini_cluster.cc:1428] Setting key 0677b36e218556aa1cb52e914ff18be6
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:48.121136 20132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:48.108609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:60639 (local address 127.11.158.190:39029)
0926 16:37:48.108754 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:48.108758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:48.109761 (+ 1003us) server_negotiation.cc:407] Connection header received
0926 16:37:48.110722 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:48.110726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:48.110780 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:48.110884 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:48.112044 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.112695 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.113829 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.114083 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.114199 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.117041 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:48.117045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:48.117048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:48.117075 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:48.118691 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:48.119317 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:48.119322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:48.119325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:48.119422 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:48.119791 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:48.119797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:48.119798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:48.120062 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:48.120169 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:48.120736 (+ 567us) server_negotiation.cc:299] Negotiation successful
0926 16:37:48.120880 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:37:48.121846 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:48.108977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:60639)
0926 16:37:48.109617 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:48.109657 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:48.110496 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:48.111060 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:48.111067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:48.111419 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:48.111892 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.111905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.112818 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.112822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.113703 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.113713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.114239 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.114243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.114426 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.114962 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:48.114978 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:48.116787 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:48.118809 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:48.118818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:48.118837 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:48.119213 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:48.119541 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:48.119543 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:48.119545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:48.119683 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:48.120165 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:48.120170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:48.120493 (+ 323us) client_negotiation.cc:769] Sending connection context
0926 16:37:48.120661 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 16:37:48.120917 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":107,"threads_started":1}
I20260926 16:37:48.123016 20418 heartbeater.cc:347] Connected to a master server at 127.11.158.190:39029
I20260926 16:37:48.123262 20418 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:48.123736 20418 heartbeater.cc:511] Master 127.11.158.190:39029 requested a full tablet report, sending...
I20260926 16:37:48.124773 20091 ts_manager.cc:194] Registered new tserver with Master: 2c63e1ab6152438f84f138c3f4941c9b (127.11.158.130:37559)
I20260926 16:37:48.125335 20091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:60639
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:37:48.144448 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39029
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:48.255614 20423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:48.255910 20423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:48.255990 20423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:48.260077 20423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:48.260169 20423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:48.260262 20423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:48.265882 20423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:48.267004 20423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:48.268071 20423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:48.278038 20429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.278090 20431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:48.278345 20423 server_base.cc:1034] running on GCE node
W20260926 16:37:48.278502 20428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:48.278976 20423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:48.279681 20423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:48.280892 20423 hybrid_clock.cc:648] HybridClock initialized: now 1790440668280873 us; error 32 us; skew 500 ppm
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:48.283622 20423 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:37:48.284550 20423 webserver.cc:460] Webserver started at http://127.11.158.131:37309/ using document root <none> and password file <none>
I20260926 16:37:48.285122 20423 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:48.285212 20423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:48.285444 20423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:37:48.287531 20423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "b2837f4995494ee39f7a0a269c780a84"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "d760d3c5c65243a31bbd5649653686a0"
server_key_iv: "84adfd3382357f92a6ffb420038bbc5e"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.288116 20423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "b2837f4995494ee39f7a0a269c780a84"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "d760d3c5c65243a31bbd5649653686a0"
server_key_iv: "84adfd3382357f92a6ffb420038bbc5e"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.292209 20423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 16:37:48.294849 20438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.296134 20423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:48.296270 20423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "b2837f4995494ee39f7a0a269c780a84"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "d760d3c5c65243a31bbd5649653686a0"
server_key_iv: "84adfd3382357f92a6ffb420038bbc5e"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.296434 20423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:48.310340 20423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:48.313537 20423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:48.313746 20423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:48.314340 20423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:48.315193 20423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:48.315265 20423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.315335 20423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:48.315389 20423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.324456 20423 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:37293
I20260926 16:37:48.324474 20551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:37293 every 8 connection(s)
I20260926 16:37:48.325384 20423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:48.330863 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20423
I20260926 16:37:48.330999 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:37:48.331296 11898 external_mini_cluster.cc:1428] Setting key fd4af9efec78698931977c634f1cac8a
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:37:48.338800 20132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:48.327210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:58619 (local address 127.11.158.190:39029)
0926 16:37:48.327381 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:48.327386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:48.328391 (+ 1005us) server_negotiation.cc:407] Connection header received
0926 16:37:48.329293 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:48.329296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:48.329344 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:48.329417 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:48.330561 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.331210 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.332682 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.332973 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.333117 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.335558 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:48.335561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:48.335563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:48.335587 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:48.337021 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:48.337483 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:48.337485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:48.337487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:48.337566 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:48.337855 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:48.337857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:48.337858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:48.338050 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:48.338132 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:48.338567 (+ 435us) server_negotiation.cc:299] Negotiation successful
0926 16:37:48.338669 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:37:48.339561 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:48.327544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:58619)
0926 16:37:48.328222 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:48.328263 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:48.329102 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:48.329589 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:48.329595 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:48.329950 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:48.330430 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.330443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.331331 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.331336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.332564 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.332574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.333132 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.333135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.333266 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.333958 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:48.333974 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:48.335409 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:37:48.337134 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:48.337138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:48.337150 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:48.337386 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:48.337649 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:37:48.337651 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:37:48.337652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:37:48.337774 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:37:48.338138 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:37:48.338143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:37:48.338355 (+ 212us) client_negotiation.cc:769] Sending connection context
0926 16:37:48.338505 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 16:37:48.338740 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":109,"threads_started":1}
I20260926 16:37:48.340740 20552 heartbeater.cc:347] Connected to a master server at 127.11.158.190:39029
I20260926 16:37:48.340987 20552 heartbeater.cc:464] Registering TS with master...
I20260926 16:37:48.341486 20552 heartbeater.cc:511] Master 127.11.158.190:39029 requested a full tablet report, sending...
I20260926 16:37:48.342505 20091 ts_manager.cc:194] Registered new tserver with Master: b2837f4995494ee39f7a0a269c780a84 (127.11.158.131:37293)
I20260926 16:37:48.343097 20091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:58619
I20260926 16:37:48.345271 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab.
Entry for principal oryx/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab.
WARNING: no policy specified for oryx/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab.
Entry for principal oryx/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab.
WARNING: no policy specified for oryx/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.158.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab.
Entry for principal oryx/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab.
I20260926 16:37:48.429199 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20155
I20260926 16:37:48.434887 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20289
I20260926 16:37:48.440203 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20423
I20260926 16:37:48.445632 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20061
I20260926 16:37:48.451879 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39029
--webserver_interface=127.11.158.190
--webserver_port=33163
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:39029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:37:48.567325 20561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:48.567618 20561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:48.567694 20561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:48.571699 20561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:37:48.571792 20561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:48.571830 20561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:48.571884 20561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:37:48.571921 20561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:37:48.577227 20561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:39029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=33163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:48.578293 20561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:48.579330 20561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:48.587828 20566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.587800 20567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:48.587843 20561 server_base.cc:1034] running on GCE node
W20260926 16:37:48.587800 20569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:48.588668 20561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:48.589761 20561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:48.590920 20561 hybrid_clock.cc:648] HybridClock initialized: now 1790440668590906 us; error 38 us; skew 500 ppm
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:48.593663 20561 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:37:48.594538 20561 webserver.cc:460] Webserver started at http://127.11.158.190:33163/ using document root <none> and password file <none>
I20260926 16:37:48.595089 20561 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:48.595193 20561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:48.599018 20561 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:48.601207 20576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.602447 20561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:37:48.602581 20561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "26c51ff5b7364414bd6fcf32445255be"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "db7b334c2849cd98aec71409e738c841"
server_key_iv: "8baf34d2026b302cad9cfba325cef8b8"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.603137 20561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:48.621971 20561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:48.625349 20561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:48.625567 20561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:48.632973 20561 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:39029
I20260926 16:37:48.633000 20628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:39029 every 8 connection(s)
I20260926 16:37:48.634047 20561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:37:48.638227 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20561
I20260926 16:37:48.638379 20629 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:37:48.638706 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:45067
--local_ip_for_outbound_sockets=127.11.158.129
--tserver_master_addrs=127.11.158.190:39029
--webserver_port=39725
--webserver_interface=127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:37:48.640113 20629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Bootstrap starting.
I20260926 16:37:48.656590 20629 log.cc:824] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Log is configured to *not* fsync() on all Append() calls
I20260926 16:37:48.663035 20629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:37:48.663419 20629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Bootstrap complete.
I20260926 16:37:48.667704 20629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:48.668128 20629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c51ff5b7364414bd6fcf32445255be, State: Initialized, Role: FOLLOWER
I20260926 16:37:48.668728 20629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:48.668870 20629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:37:48.668953 20629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:37:48.669065 20629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:37:48.670953 20629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:48.671289 20629 leader_election.cc:302] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c51ff5b7364414bd6fcf32445255be; no voters:
I20260926 16:37:48.671613 20629 leader_election.cc:288] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:37:48.671738 20633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:37:48.672008 20633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [term 2 LEADER]: Becoming Leader. State: Replica: 26c51ff5b7364414bd6fcf32445255be, State: Running, Role: LEADER
I20260926 16:37:48.672379 20633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } }
I20260926 16:37:48.672899 20629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:37:48.673871 20634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "26c51ff5b7364414bd6fcf32445255be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } } }
I20260926 16:37:48.673898 20635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c51ff5b7364414bd6fcf32445255be. Latest consensus state: current_term: 2 leader_uuid: "26c51ff5b7364414bd6fcf32445255be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51ff5b7364414bd6fcf32445255be" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39029 } } }
I20260926 16:37:48.674012 20634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:48.674014 20635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be [sys.catalog]: This master's current role is: LEADER
I20260926 16:37:48.674409 20642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:37:48.677670 20642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:37:48.678319 20642 catalog_manager.cc:1259] Loaded cluster ID: 7e3866c74c994131832357c263028bff
I20260926 16:37:48.678380 20642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:37:48.680670 20642 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:37:48.681294 20642 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26c51ff5b7364414bd6fcf32445255be: Loaded TSK: 0
I20260926 16:37:48.682513 20642 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:37:48.754280 20631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:48.754557 20631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:48.754603 20631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:48.758733 20631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:48.758797 20631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:48.758858 20631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:37:48.764835 20631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.129.keytab
--principal=oryx/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:45067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=39725
--tserver_master_addrs=127.11.158.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:48.765872 20631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:48.766882 20631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:48.776957 20655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.776947 20658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.777410 20656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:48.777449 20631 server_base.cc:1034] running on GCE node
I20260926 16:37:48.777846 20631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:48.778529 20631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:48.779721 20631 hybrid_clock.cc:648] HybridClock initialized: now 1790440668779694 us; error 49 us; skew 500 ppm
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440668, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:48.782725 20631 init.cc:377] Logged in from keytab as oryx/127.11.158.129@KRBTEST.COM (short username oryx)
I20260926 16:37:48.783638 20631 webserver.cc:460] Webserver started at http://127.11.158.129:39725/ using document root <none> and password file <none>
I20260926 16:37:48.784209 20631 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:48.784317 20631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:48.788313 20631 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:48.790704 20665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.792053 20631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 16:37:48.792203 20631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "4ac584f51bde4ed095deb9dd8faccabd"
format_stamp: "Formatted at 2026-09-26 16:37:47 on dist-test-slave-p39x"
server_key: "a6e8d7ff7ef8a3be1ef831d712c0813d"
server_key_iv: "a03b0f38ff0974dd0bc91a9a922ad491"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.792826 20631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:48.803217 20631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:48.806738 20631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:48.806946 20631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:48.807596 20631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:48.808547 20631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:48.808611 20631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.808686 20631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:48.808732 20631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.820909 20631 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:45067
I20260926 16:37:48.820930 20778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:45067 every 8 connection(s)
I20260926 16:37:48.822068 20631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:37:48.825217 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20631
I20260926 16:37:48.825704 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:37559
--local_ip_for_outbound_sockets=127.11.158.130
--tserver_master_addrs=127.11.158.190:39029
--webserver_port=34295
--webserver_interface=127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:37:48.836290 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:48.824724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:44139 (local address 127.11.158.190:39029)
0926 16:37:48.825301 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:48.825326 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:48.825912 (+ 586us) server_negotiation.cc:407] Connection header received
0926 16:37:48.826973 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:48.826984 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:48.827284 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:48.827615 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:48.828836 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.829878 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.831030 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.831511 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.831689 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.835495 (+ 3806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:44139: BlockingRecv error: recv got EOF from 127.11.158.129:44139 (error 108)
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":138,"threads_started":1}
I20260926 16:37:48.836300 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:48.824930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:44139)
0926 16:37:48.825687 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:48.825741 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:48.826541 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:48.827692 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:48.827700 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:48.828125 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:48.828668 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.828683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.830016 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.830020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.830901 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.830910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.831681 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.831685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.831867 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.832535 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:48.832552 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:48.835269 (+ 2717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":144,"threads_started":1}
W20260926 16:37:48.837021 20779 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:39029 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.158.190:39029: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:48.843271 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:48.837469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:49959)
0926 16:37:48.837666 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:48.837678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:48.837746 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:48.837982 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:48.837984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:48.838131 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:48.838226 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.838230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.839302 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.839307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.839894 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.839904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.840605 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.840608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.840684 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.841201 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:48.841213 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:48.843156 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123}
W20260926 16:37:48.843451 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:48.837576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:49959 (local address 127.11.158.190:39029)
0926 16:37:48.837723 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:48.837727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:48.837743 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:48.837820 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:48.837823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:48.837872 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:48.837968 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:48.838322 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.839157 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.840065 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.840491 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.840609 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.843331 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:49959: BlockingRecv error: recv got EOF from 127.11.158.129:49959 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:48.849548 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:48.843888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:40873)
0926 16:37:48.844150 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:48.844168 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:48.844290 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:48.844631 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:48.844634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:48.844815 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:48.844894 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.844900 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.845676 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.845682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.846267 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:48.846275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.846746 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.846750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:48.846840 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.847362 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:48.847374 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:48.849338 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":127,"threads_started":1}
W20260926 16:37:48.849586 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:48.843961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:40873 (local address 127.11.158.190:39029)
0926 16:37:48.844104 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:48.844118 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:48.844248 (+ 130us) server_negotiation.cc:407] Connection header received
0926 16:37:48.844451 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:48.844460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:48.844510 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:48.844604 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:48.845018 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.845564 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.846398 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:48.846635 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:48.846747 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:48.849474 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:40873: BlockingRecv error: recv got EOF from 127.11.158.129:40873 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 16:37:48.849869 20779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:37:48.942765 20783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:48.943080 20783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:48.943131 20783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:48.947717 20783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:48.947806 20783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:48.947870 20783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:37:48.953608 20783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.130.keytab
--principal=oryx/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:37559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=34295
--tserver_master_addrs=127.11.158.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:48.954762 20783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:48.955848 20783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:37:48.965602 20783 server_base.cc:1034] running on GCE node
W20260926 16:37:48.965543 20792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.965548 20789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:48.965548 20790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:48.966241 20783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:48.966938 20783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:48.968132 20783 hybrid_clock.cc:648] HybridClock initialized: now 1790440668968099 us; error 47 us; skew 500 ppm
Sep 26 16:37:48 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440668, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:48.970930 20783 init.cc:377] Logged in from keytab as oryx/127.11.158.130@KRBTEST.COM (short username oryx)
I20260926 16:37:48.971822 20783 webserver.cc:460] Webserver started at http://127.11.158.130:34295/ using document root <none> and password file <none>
I20260926 16:37:48.972517 20783 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:48.972635 20783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:48.977005 20783 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:37:48.979176 20799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:48.980512 20783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:37:48.980633 20783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "2c63e1ab6152438f84f138c3f4941c9b"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "2c5d99440baf7c80369f04bb65dba1cc"
server_key_iv: "481a64f6afdd36bccb4ae022cff3ce36"
server_key_version: "encryptionkey@0"
I20260926 16:37:48.981185 20783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:49.014621 20783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:49.017630 20783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:49.017838 20783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:49.018453 20783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:49.019322 20783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:49.019369 20783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:49.019431 20783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:49.019464 20783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:49.028508 20783 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:37559
I20260926 16:37:49.028542 20912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:37559 every 8 connection(s)
I20260926 16:37:49.029603 20783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:37:49.032785 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20783
I20260926 16:37:49.033306 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:37293
--local_ip_for_outbound_sockets=127.11.158.131
--tserver_master_addrs=127.11.158.190:39029
--webserver_port=37309
--webserver_interface=127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:37:49.043418 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:49.031533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:52839 (local address 127.11.158.190:39029)
0926 16:37:49.031668 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.031672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.032894 (+ 1222us) server_negotiation.cc:407] Connection header received
0926 16:37:49.034301 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.034305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.034360 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.034442 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.035825 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.036455 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.038486 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.038745 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.038855 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.043308 (+ 4453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:52839: BlockingRecv error: recv got EOF from 127.11.158.130:52839 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:37:49.044420 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.031902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:52839)
0926 16:37:49.032705 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:49.032766 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:49.034014 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:49.034683 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:49.034698 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:49.035135 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:49.035647 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.035661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.036599 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.036605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.038321 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.038339 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.038938 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.038945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.039203 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.039950 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:49.039980 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:49.043157 (+ 3177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":104,"threads_started":1}
W20260926 16:37:49.045054 20913 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:39029 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.158.190:39029: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:49.051369 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.045525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:32895)
0926 16:37:49.045761 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:49.045773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:49.045845 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:49.046103 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:49.046105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:49.046254 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:49.046357 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.046362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.047482 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.047485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.048142 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.048152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.048763 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.048766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.048849 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.049366 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:49.049378 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:49.051230 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166}
W20260926 16:37:49.051661 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:49.045652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:32895 (local address 127.11.158.190:39029)
0926 16:37:49.045780 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.045785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.045804 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:37:49.045934 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.045937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.045985 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.046068 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.046457 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.047317 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.048263 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.048652 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.048752 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.051365 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:32895: BlockingRecv error: recv got EOF from 127.11.158.130:32895 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:49.057513 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.051919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:54865)
0926 16:37:49.052040 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:49.052056 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:49.052133 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:49.052489 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:49.052492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:49.052665 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:49.052755 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.052763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.053475 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.053479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.054346 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.054356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.054788 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.054793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.054883 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.055330 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:49.055346 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:49.057371 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 16:37:49.057843 20913 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:37:49.058027 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:49.052146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:54865 (local address 127.11.158.190:39029)
0926 16:37:49.052260 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.052263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.052275 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:37:49.052308 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.052309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.052386 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.052458 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.052871 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.053376 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.054472 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.054682 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.054774 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.057522 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:54865: BlockingRecv error: recv got EOF from 127.11.158.130:54865 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 16:37:49.153017 20916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:37:49.153309 20916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:37:49.153353 20916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:37:49.157555 20916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:37:49.157622 20916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:37:49.157684 20916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:37:49.163367 20916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/krb5kdc/oryx_127.11.158.131.keytab
--principal=oryx/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:37293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=37309
--tserver_master_addrs=127.11.158.190:39029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:37:49.164448 20916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:37:49.165468 20916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:37:49.175271 20924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:49.175315 20921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:37:49.175278 20922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:37:49.175417 20916 server_base.cc:1034] running on GCE node
I20260926 16:37:49.176023 20916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:37:49.176705 20916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:37:49.177877 20916 hybrid_clock.cc:648] HybridClock initialized: now 1790440669177842 us; error 48 us; skew 500 ppm
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440669, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:37:49.180699 20916 init.cc:377] Logged in from keytab as oryx/127.11.158.131@KRBTEST.COM (short username oryx)
I20260926 16:37:49.181560 20916 webserver.cc:460] Webserver started at http://127.11.158.131:37309/ using document root <none> and password file <none>
I20260926 16:37:49.182119 20916 fs_manager.cc:359] Metadata directory not provided
I20260926 16:37:49.182231 20916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:37:49.186312 20916 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260926 16:37:49.188400 20931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:49.189684 20916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:37:49.189839 20916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "b2837f4995494ee39f7a0a269c780a84"
format_stamp: "Formatted at 2026-09-26 16:37:48 on dist-test-slave-p39x"
server_key: "d760d3c5c65243a31bbd5649653686a0"
server_key_iv: "84adfd3382357f92a6ffb420038bbc5e"
server_key_version: "encryptionkey@0"
I20260926 16:37:49.190426 20916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:37:49.201925 20916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:37:49.205030 20916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:37:49.205232 20916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:37:49.205811 20916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:37:49.206655 20916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:37:49.206720 20916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:49.206786 20916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:37:49.206831 20916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:37:49.216631 20916 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:37293
I20260926 16:37:49.216647 21044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:37293 every 8 connection(s)
I20260926 16:37:49.217644 20916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:37:49.220647 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 20916
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:37:49.229770 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:49.219448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:39151 (local address 127.11.158.190:39029)
0926 16:37:49.219587 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.219590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.220606 (+ 1016us) server_negotiation.cc:407] Connection header received
0926 16:37:49.222028 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.222031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.222078 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.222169 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.223448 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.224169 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.225723 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.226079 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.226217 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.229647 (+ 3430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:39151: BlockingRecv error: recv got EOF from 127.11.158.131:39151 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:37:49.230495 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.219771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:39151)
0926 16:37:49.220456 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:49.220496 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:49.221799 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:49.222341 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:49.222347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:49.222690 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:49.223308 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.223326 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.224292 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.224295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.225589 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.225600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.226227 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.226230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.226364 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.226901 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:49.226916 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:49.229526 (+ 2610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":111,"threads_started":1}
I20260926 16:37:49.230921 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.221586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48518 (local address 127.11.158.190:39029)
0926 16:37:49.221805 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.221819 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.221833 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:37:49.221877 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.221880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.221929 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.222033 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.222787 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.223545 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.224632 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.224963 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.225073 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.226713 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:37:49.226719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:37:49.226732 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:37:49.226756 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:37:49.228785 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:49.229204 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:49.229212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:49.229218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:49.229372 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:37:49.229697 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:37:49.229700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:37:49.229702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:37:49.230128 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:37:49.230315 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 16:37:49.230554 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 16:37:49.230785 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
W20260926 16:37:49.231017 21045 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:39029 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.158.190:39029: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:49.236675 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.231412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:54619)
0926 16:37:49.231504 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:49.231515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:49.231571 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:49.232161 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:49.232163 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:49.232292 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:49.232434 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.232439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.233321 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.233324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.233959 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.233968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.234356 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.234358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.234423 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.234800 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:49.234810 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:49.236583 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 16:37:49.236850 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:49.231797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:54619 (local address 127.11.158.190:39029)
0926 16:37:49.231934 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.231938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.231953 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:37:49.232000 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.232003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.232041 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.232125 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.232531 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.233201 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.234067 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.234265 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.234343 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.236724 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:54619: BlockingRecv error: recv got EOF from 127.11.158.131:54619 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:49.241658 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.237335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:37069)
0926 16:37:49.237450 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:49.237465 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:49.237538 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:49.237725 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:49.237727 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:49.237842 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:49.237906 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.237910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.238532 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.238534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.239028 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.239034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.239421 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.239423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.239476 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.239879 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:49.239890 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:49.241565 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:37:49.241865 20781 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:49.237404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:37069 (local address 127.11.158.190:39029)
0926 16:37:49.237520 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.237523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.237535 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:37:49.237607 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.237609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.237640 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.237701 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.237989 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.238450 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.239128 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.239322 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.239395 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.241728 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:37069: BlockingRecv error: recv got EOF from 127.11.158.131:37069 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 16:37:49.241901 21045 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:49 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:49.856523 21050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:49.850705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:50159)
0926 16:37:49.851068 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:49.851097 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:49.851209 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:49.851501 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:49.851507 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:49.851740 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:49.851898 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.851907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.852891 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.852894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.853485 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:49.853494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.853922 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.853925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:49.853999 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.854497 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:49.854508 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:49.856392 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
W20260926 16:37:49.856729 21051 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:49.850935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:50159 (local address 127.11.158.190:39029)
0926 16:37:49.851206 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:49.851212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:49.851228 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:49.851291 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:49.851294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:49.851335 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:49.851450 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:49.852042 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.852762 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.853601 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:49.853818 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:49.853901 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:49.856549 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:50159: BlockingRecv error: recv got EOF from 127.11.158.129:50159 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
Sep 26 16:37:50 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:50 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:50.063830 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:50.058503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:43343)
0926 16:37:50.058781 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:50.058795 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:50.058894 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:50.059131 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:50.059133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:50.059308 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:50.059423 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:50.059428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.060189 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.060192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:50.060868 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:50.060877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.061322 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.061325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:50.061398 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:50.061883 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:50.061894 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:50.063709 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
W20260926 16:37:50.064050 21051 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:50.058640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:43343 (local address 127.11.158.190:39029)
0926 16:37:50.058781 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:50.058790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:50.058861 (+ 71us) server_negotiation.cc:407] Connection header received
0926 16:37:50.058973 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:50.058975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:50.059018 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:50.059095 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:50.059555 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.060077 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.060989 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.061221 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.061304 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:50.063892 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:43343: BlockingRecv error: recv got EOF from 127.11.158.130:43343 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 16:37:50 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:50 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:50.249109 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:50.242724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:55913)
0926 16:37:50.243017 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:50.243031 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:50.243173 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:50.243617 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:50.243631 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:50.243826 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:50.243946 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:50.243953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.244927 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.244932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:50.245810 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:50.245823 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.246449 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.246452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:50.246559 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:50.247109 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:50.247120 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:50.248975 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":76,"threads_started":1}
W20260926 16:37:50.249241 21051 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:50.242832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:55913 (local address 127.11.158.190:39029)
0926 16:37:50.243031 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:50.243035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:50.243129 (+ 94us) server_negotiation.cc:407] Connection header received
0926 16:37:50.243342 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:50.243348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:50.243414 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:50.243538 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:50.244104 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.244784 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.245965 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.246284 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.246370 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:50.249125 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:55913: BlockingRecv error: recv got EOF from 127.11.158.131:55913 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 16:37:50 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:50 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:50.862480 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:50.857296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:42927)
0926 16:37:50.857593 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:50.857608 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:50.857710 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:50.857955 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:50.857957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:50.858158 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:50.858258 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:50.858263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.859054 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.859057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:50.859598 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:50.859605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.860012 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.860015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:50.860090 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:50.860619 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:50.860631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:50.862361 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
W20260926 16:37:50.862735 21055 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:50.857438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:42927 (local address 127.11.158.190:39029)
0926 16:37:50.857692 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:50.857696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:50.857709 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:50.857799 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:50.857801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:50.857845 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:50.857922 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:50.858394 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.858947 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.859695 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:50.859915 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:50.859990 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:50.862566 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:42927: BlockingRecv error: recv got EOF from 127.11.158.129:42927 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
Sep 26 16:37:51 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:51 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:51.069736 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:51.064692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:42271)
0926 16:37:51.064975 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:51.064988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:51.065084 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:51.065305 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:51.065307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:51.065478 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:51.065568 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:51.065573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.066307 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.066309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:51.066877 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:51.066884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.067312 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.067314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:51.067370 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:51.067845 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:51.067856 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:51.069609 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
W20260926 16:37:51.069931 21055 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:51.064840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:42271 (local address 127.11.158.190:39029)
0926 16:37:51.064993 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:51.064996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:51.065052 (+ 56us) server_negotiation.cc:407] Connection header received
0926 16:37:51.065161 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:51.065164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:51.065205 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:51.065284 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:51.065697 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.066216 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.066988 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.067213 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.067298 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:51.069763 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:42271: BlockingRecv error: recv got EOF from 127.11.158.130:42271 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 16:37:51 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:51 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:51.255385 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:51.250016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:33637)
0926 16:37:51.250272 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:51.250285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:51.250371 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:51.250626 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:51.250629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:51.250817 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:51.250910 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:51.250915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.251683 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.251686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:51.252242 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:51.252249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.252895 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.252899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:51.252963 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:51.253457 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:51.253468 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:51.255256 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":68,"threads_started":1}
W20260926 16:37:51.255558 21055 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:51.250167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:33637 (local address 127.11.158.190:39029)
0926 16:37:51.250321 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:51.250325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:51.250339 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:37:51.250457 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:51.250460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:51.250514 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:51.250586 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:51.251037 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.251557 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.252523 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.252779 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.252867 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:51.255409 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:33637: BlockingRecv error: recv got EOF from 127.11.158.131:33637 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 26 16:37:51 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:51 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:51.868598 21058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:51.863318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:35813)
0926 16:37:51.863600 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:51.863613 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:51.863755 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:51.864072 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:51.864075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:51.864269 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:51.864395 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:51.864401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.865125 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.865127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:51.865689 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:51.865696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.866089 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.866092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:51.866158 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:51.866681 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:51.866693 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:51.868426 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
W20260926 16:37:51.868757 21059 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:51.863369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:35813 (local address 127.11.158.190:39029)
0926 16:37:51.863606 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:51.863613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:51.863696 (+ 83us) server_negotiation.cc:407] Connection header received
0926 16:37:51.863868 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:51.863871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:51.863926 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:51.864025 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:51.864492 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.865028 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.865781 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:51.865993 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:51.866069 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:51.868613 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:35813: BlockingRecv error: recv got EOF from 127.11.158.129:35813 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
Sep 26 16:37:52 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:52 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:52.076771 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:52.070560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:33659)
0926 16:37:52.070885 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:52.070901 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:52.071024 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:52.071308 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:52.071311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:52.071517 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:52.071630 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:52.071635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.072434 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.072441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:52.073132 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:52.073141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.073626 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.073630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:52.073716 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:52.074383 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:52.074400 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:52.076638 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
W20260926 16:37:52.076910 21059 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:52.070721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:33659 (local address 127.11.158.190:39029)
0926 16:37:52.070890 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:52.070897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:52.070965 (+ 68us) server_negotiation.cc:407] Connection header received
0926 16:37:52.071138 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:52.071142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:52.071194 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:52.071281 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:52.071750 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.072291 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.073249 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.073510 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.073621 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:52.076765 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:33659: BlockingRecv error: recv got EOF from 127.11.158.130:33659 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 16:37:52 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:52 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:52.261458 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:52.256228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:56637)
0926 16:37:52.256500 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:52.256518 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:52.256641 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:52.256917 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:52.256920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:52.257095 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:52.257211 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:52.257217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.257968 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.257971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:52.258535 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:52.258543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.258927 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.258930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:52.258987 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:52.259485 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:52.259497 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:52.261328 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":130,"threads_started":1}
W20260926 16:37:52.261685 21059 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:52.256362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:56637 (local address 127.11.158.190:39029)
0926 16:37:52.256498 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:52.256501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:52.256603 (+ 102us) server_negotiation.cc:407] Connection header received
0926 16:37:52.256726 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:52.256728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:52.256773 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:52.256850 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:52.257327 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.257874 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.258630 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.258836 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.258911 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:52.261522 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:56637: BlockingRecv error: recv got EOF from 127.11.158.131:56637 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 16:37:52 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:52 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:52.875142 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:52.869456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:49955)
0926 16:37:52.869741 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:52.869754 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:52.869849 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:52.870063 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:52.870065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:52.870241 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:52.870335 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:52.870340 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.871202 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.871208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:52.871797 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:52.871806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.872295 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.872299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:52.872412 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:52.872986 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:52.872999 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:52.874971 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":85,"threads_started":1}
W20260926 16:37:52.875252 21062 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:52.869472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:49955 (local address 127.11.158.190:39029)
0926 16:37:52.869737 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:52.869741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:52.869801 (+ 60us) server_negotiation.cc:407] Connection header received
0926 16:37:52.869914 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:52.869916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:52.869956 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:52.870034 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:52.870429 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.871004 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.871923 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:52.872166 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:52.872290 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:52.875103 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:49955: BlockingRecv error: recv got EOF from 127.11.158.129:49955 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
Sep 26 16:37:53 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:53 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:53.083619 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:53.077568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:60041)
0926 16:37:53.077830 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:53.077843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:53.077945 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:53.078175 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:53.078177 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:53.078350 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:53.078442 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:53.078447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.079219 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.079222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:53.079804 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:53.079814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.080400 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.080406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:53.080523 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:53.081151 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:53.081170 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:53.083458 (+ 2288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
W20260926 16:37:53.083740 21062 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:53.077659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:60041 (local address 127.11.158.190:39029)
0926 16:37:53.077792 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:53.077796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:53.077932 (+ 136us) server_negotiation.cc:407] Connection header received
0926 16:37:53.078016 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:53.078019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:53.078068 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:53.078145 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:53.078545 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.079087 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.079962 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.080229 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.080353 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:53.083625 (+ 3272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:60041: BlockingRecv error: recv got EOF from 127.11.158.130:60041 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 16:37:53 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:53 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:37:53.268956 21062 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:53.262345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:50961 (local address 127.11.158.190:39029)
0926 16:37:53.262487 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:53.262492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:53.262616 (+ 124us) server_negotiation.cc:407] Connection header received
0926 16:37:53.262775 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:53.262777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:53.262825 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:53.262890 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:53.263536 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.264100 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.265447 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.265702 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.265787 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:53.268801 (+ 3014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:50961: BlockingRecv error: recv got EOF from 127.11.158.131:50961 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:37:53.269022 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:53.262214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:50961)
0926 16:37:53.262515 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:53.262530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:53.262670 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:53.262957 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:53.262961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:53.263235 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:53.263377 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:53.263387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.264228 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.264233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:53.265309 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:53.265324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.265819 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.265824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:53.265936 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:53.266466 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:53.266485 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:53.268668 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":83,"threads_started":1}
Sep 26 16:37:53 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:53 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:53.881829 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:53.875930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:47833)
0926 16:37:53.876205 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:53.876223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:53.876321 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:53.876637 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:53.876639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:53.876820 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:53.876927 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:53.876933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.877771 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.877774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:53.878348 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:53.878357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.878869 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.878872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:53.878944 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:53.879525 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:53.879538 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:53.881694 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
W20260926 16:37:53.882057 21067 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:53.876084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:47833 (local address 127.11.158.190:39029)
0926 16:37:53.876323 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:53.876327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:53.876378 (+ 51us) server_negotiation.cc:407] Connection header received
0926 16:37:53.876453 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:53.876455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:53.876495 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:53.876570 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:53.877049 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.877586 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.878491 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:53.878736 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:53.878820 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:53.881897 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:47833: BlockingRecv error: recv got EOF from 127.11.158.129:47833 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":74,"threads_started":1}
Sep 26 16:37:54 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:54 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:54.091318 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:54.084508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:57791)
0926 16:37:54.084819 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:54.084836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:54.084967 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:54.085279 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:54.085282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:54.085489 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:54.085582 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:54.085587 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.086711 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.086718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:54.087464 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:54.087480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.088182 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.088187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:54.088281 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:54.088932 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:54.088960 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:54.091141 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
W20260926 16:37:54.091410 21067 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:54.084633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:57791 (local address 127.11.158.190:39029)
0926 16:37:54.084886 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:54.084892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:54.084909 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:37:54.085083 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:54.085086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:54.085142 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:54.085231 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:54.085702 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.086585 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.087604 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.088044 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.088151 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:54.091291 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:57791: BlockingRecv error: recv got EOF from 127.11.158.130:57791 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Sep 26 16:37:54 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:54 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:54.275949 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:54.269966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:56761)
0926 16:37:54.270246 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:54.270260 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:54.270354 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:54.270696 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:54.270699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:54.270880 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:54.270975 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:54.270981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.271732 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.271734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:54.272310 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:54.272317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.272934 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.272938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:54.273017 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:54.273763 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:54.273783 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:54.275819 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
W20260926 16:37:54.276153 21067 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:54.270108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:56761 (local address 127.11.158.190:39029)
0926 16:37:54.270291 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:54.270295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:54.270313 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:37:54.270488 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:54.270491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:54.270544 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:54.270629 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:54.271099 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.271622 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.272510 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.272791 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.272921 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:54.275969 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:56761: BlockingRecv error: recv got EOF from 127.11.158.131:56761 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 26 16:37:54 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:54 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:54.888178 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:54.882622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:47915)
0926 16:37:54.882912 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:54.882926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:54.883022 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:54.883309 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:54.883311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:54.883513 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:54.883609 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:54.883614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.884396 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.884402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:54.885034 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:54.885045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.885601 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.885604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:54.885670 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:54.886160 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:54.886172 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:54.888047 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":69,"threads_started":1}
W20260926 16:37:54.888486 21071 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:54.882705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:47915 (local address 127.11.158.190:39029)
0926 16:37:54.882967 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:54.882970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:54.882983 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:54.883139 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:54.883142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:54.883191 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:54.883273 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:54.883709 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.884257 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.885158 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:54.885463 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:54.885572 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:54.888280 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:47915: BlockingRecv error: recv got EOF from 127.11.158.129:47915 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":100,"threads_started":1}
Sep 26 16:37:55 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:55 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:55.098582 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:55.092239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:58117)
0926 16:37:55.092718 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:55.092739 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:55.092875 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:55.093268 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:55.093271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:55.093535 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:55.093714 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:55.093724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.094781 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.094784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:55.095467 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:55.095476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.095933 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.095936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:55.096004 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:55.096533 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:55.096544 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:55.098465 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":115,"threads_started":1}
W20260926 16:37:55.098707 21071 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:55.092262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:58117 (local address 127.11.158.190:39029)
0926 16:37:55.092515 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:55.092521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:55.092834 (+ 313us) server_negotiation.cc:407] Connection header received
0926 16:37:55.093049 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:55.093052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:55.093107 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:55.093183 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:55.093836 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.094599 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.095586 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.095820 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.095898 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:55.098579 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:58117: BlockingRecv error: recv got EOF from 127.11.158.130:58117 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Sep 26 16:37:55 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:55 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:55.282518 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:55.276925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:38471)
0926 16:37:55.277224 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:55.277237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:55.277352 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:55.277631 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:55.277634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:55.277829 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:55.277924 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:55.277930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.278706 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.278708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:55.279255 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:55.279262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.279707 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.279709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:55.279773 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:55.280353 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:55.280368 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:55.282393 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":106,"threads_started":1}
W20260926 16:37:55.282680 21071 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:55.277001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:38471 (local address 127.11.158.190:39029)
0926 16:37:55.277158 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:55.277163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:55.277323 (+ 160us) server_negotiation.cc:407] Connection header received
0926 16:37:55.277438 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:55.277441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:55.277490 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:55.277569 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:55.278043 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.278603 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.279363 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.279599 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.279703 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:55.282544 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:38471: BlockingRecv error: recv got EOF from 127.11.158.131:38471 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 16:37:55 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:55 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:55.894968 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:55.889121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:39137)
0926 16:37:55.889479 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:55.889512 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:55.889629 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:55.889964 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:55.889968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:55.890212 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:55.890314 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:55.890321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.891104 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.891107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:55.891681 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:55.891690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.892162 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.892165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:55.892238 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:55.892806 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:55.892818 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:55.894744 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":84,"threads_started":1}
W20260926 16:37:55.895004 21075 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:55.889247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:39137 (local address 127.11.158.190:39029)
0926 16:37:55.889576 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:55.889580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:55.889592 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:37:55.889769 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:55.889772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:55.889826 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:55.889929 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:55.890430 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.890968 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.891837 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:55.892063 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:55.892148 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:55.894885 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:39137: BlockingRecv error: recv got EOF from 127.11.158.129:39137 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":74,"threads_started":1}
Sep 26 16:37:56 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:56 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:56.105556 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:56.099566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:52095)
0926 16:37:56.099893 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:56.099907 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:56.100049 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:56.100308 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:56.100310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:56.100539 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:56.100632 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:56.100637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.101402 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.101407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:56.101982 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:56.101990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.102468 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.102472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:56.102552 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:56.103137 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:56.103149 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:56.105394 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":97,"threads_started":1}
W20260926 16:37:56.105721 21075 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:56.099892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:52095 (local address 127.11.158.190:39029)
0926 16:37:56.100040 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:56.100045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:56.100061 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:56.100148 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:56.100152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:56.100197 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:56.100288 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:56.100748 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.101290 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.102110 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.102355 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.102465 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:56.105593 (+ 3128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:52095: BlockingRecv error: recv got EOF from 127.11.158.130:52095 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 16:37:56 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:56 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:56.289397 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:56.283580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:34421)
0926 16:37:56.283925 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:56.283940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:56.284061 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:56.284445 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:56.284449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:56.284700 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:56.284793 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:56.284799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.285674 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.285677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:56.286326 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:56.286336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.286795 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.286799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:56.286867 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:56.287327 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:56.287338 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:56.289280 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":110,"threads_started":1}
W20260926 16:37:56.289544 21075 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:56.283637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:34421 (local address 127.11.158.190:39029)
0926 16:37:56.283789 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:56.283794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:56.284011 (+ 217us) server_negotiation.cc:407] Connection header received
0926 16:37:56.284131 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:56.284133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:56.284227 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:56.284306 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:56.284917 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.285555 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.286430 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.286662 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.286745 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:56.289422 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:34421: BlockingRecv error: recv got EOF from 127.11.158.131:34421 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 16:37:56 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:56 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:56.902544 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:56.895872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:37083)
0926 16:37:56.896147 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:56.896161 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:56.896284 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:56.896651 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:56.896656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:56.896893 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:56.897025 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:56.897033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.897867 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.897869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:56.898441 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:56.898448 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.898899 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.898902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:56.898967 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:56.899440 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:56.899450 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:56.902415 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
W20260926 16:37:56.902927 21079 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:56.896029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:37083 (local address 127.11.158.190:39029)
0926 16:37:56.896245 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:56.896252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:56.896269 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:37:56.896439 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:56.896443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:56.896498 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:56.896622 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:56.897162 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.897761 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.898563 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:56.898790 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:56.898869 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:56.902601 (+ 3732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:37083: BlockingRecv error: recv got EOF from 127.11.158.129:37083 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
Sep 26 16:37:57 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:57 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:57.112257 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:57.106630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:49079)
0926 16:37:57.106886 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:57.106901 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:57.107019 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:57.107278 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:57.107280 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:57.107451 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:57.107573 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:57.107578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.108554 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.108558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:57.109120 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:57.109130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.109554 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.109558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:57.109648 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:57.110237 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:57.110249 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:57.112071 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
W20260926 16:37:57.112327 21079 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:57.106822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:49079 (local address 127.11.158.190:39029)
0926 16:37:57.106965 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:57.106969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:57.106986 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:37:57.107106 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:57.107109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:57.107164 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:57.107249 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:57.107677 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.108446 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.109224 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.109432 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.109544 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:57.112219 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:49079: BlockingRecv error: recv got EOF from 127.11.158.130:49079 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 16:37:57 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:57 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:57.295866 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:57.290286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:59151)
0926 16:37:57.290610 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:57.290628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:57.290750 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:57.290978 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:57.290980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:57.291160 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:57.291263 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:57.291269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.292005 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.292009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:57.292645 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:57.292655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.293100 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.293103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:57.293174 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:57.293850 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:57.293861 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:57.295743 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
W20260926 16:37:57.295992 21079 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:57.290346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:59151 (local address 127.11.158.190:39029)
0926 16:37:57.290489 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:57.290494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:57.290701 (+ 207us) server_negotiation.cc:407] Connection header received
0926 16:37:57.290829 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:57.290831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:57.290877 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:57.290946 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:57.291361 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.291891 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.292763 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.292976 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.293059 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:57.295894 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:59151: BlockingRecv error: recv got EOF from 127.11.158.131:59151 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 16:37:57 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:57 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:57.912789 21082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:57.903505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:37529)
0926 16:37:57.903784 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:57.903803 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:57.903912 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:57.904232 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:57.904235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:57.904468 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:57.904566 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:57.904572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.905340 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.905346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:57.905916 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:57.905923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.906396 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.906399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:57.906486 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:57.910686 (+ 4200us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:57.910700 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:57.912615 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
W20260926 16:37:57.912902 21083 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:57.903561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:37529 (local address 127.11.158.190:39029)
0926 16:37:57.903852 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:57.903857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:57.903872 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:37:57.904050 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:57.904054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:57.904114 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:57.904208 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:57.904688 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.905232 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.906035 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:57.906296 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:57.906393 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:57.912768 (+ 6375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:37529: BlockingRecv error: recv got EOF from 127.11.158.129:37529 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
Sep 26 16:37:58 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:58 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:58.118983 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:58.113205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:60541)
0926 16:37:58.113472 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:58.113486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:58.113589 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:58.113844 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:58.113846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:58.114024 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:58.114115 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:58.114120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.114865 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.114869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:58.115675 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:58.115689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.116266 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.116268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:58.116409 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:58.116924 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:58.116936 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:58.118809 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
W20260926 16:37:58.119066 21083 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:58.113278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:60541 (local address 127.11.158.190:39029)
0926 16:37:58.113428 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:58.113434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:58.113557 (+ 123us) server_negotiation.cc:407] Connection header received
0926 16:37:58.113689 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:58.113691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:58.113742 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:58.113811 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:58.114217 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.114741 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.115807 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.116125 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.116249 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:58.118939 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:60541: BlockingRecv error: recv got EOF from 127.11.158.130:60541 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 16:37:58 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:58 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:58.303346 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:58.296788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:38185)
0926 16:37:58.297234 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:58.297250 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:58.297414 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:58.297740 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:58.297744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:58.297996 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:58.298149 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:58.298158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.299205 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.299208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:58.299758 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:58.299766 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.300151 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.300154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:58.300222 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:58.300719 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:58.300734 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:58.303175 (+ 2441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":94,"threads_started":1}
W20260926 16:37:58.303392 21083 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:58.296921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:38185 (local address 127.11.158.190:39029)
0926 16:37:58.297071 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:58.297077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:58.297326 (+ 249us) server_negotiation.cc:407] Connection header received
0926 16:37:58.297550 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:58.297555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:58.297613 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:58.297700 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:58.298278 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.299088 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.299855 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.300061 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.300148 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:58.303290 (+ 3142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:38185: BlockingRecv error: recv got EOF from 127.11.158.131:38185 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 16:37:58 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:58 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:58.919124 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:58.913695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:58155)
0926 16:37:58.913989 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:58.914003 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:58.914103 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:58.914393 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:58.914395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:58.914574 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:58.914670 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:58.914675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.915492 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.915494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:58.916065 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:58.916075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.916577 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.916581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:58.916641 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:58.917194 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:58.917205 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:58.918994 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":80,"threads_started":1}
W20260926 16:37:58.919277 21087 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:58.913737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:58155 (local address 127.11.158.190:39029)
0926 16:37:58.914029 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:58.914033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:58.914049 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:37:58.914224 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:58.914227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:58.914274 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:58.914359 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:58.914840 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.915385 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.916204 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:58.916469 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:58.916553 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:58.919145 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:58155: BlockingRecv error: recv got EOF from 127.11.158.129:58155 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":100,"threads_started":1}
Sep 26 16:37:59 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:59 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:59.125916 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:59.119982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:59745)
0926 16:37:59.120287 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:59.120310 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:59.120467 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:59.120764 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:59.120767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:59.120981 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:59.121071 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:59.121076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.121863 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.121870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:59.122452 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:59.122461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.122939 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.122942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:59.123027 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:59.123609 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:59.123620 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:59.125773 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":87,"threads_started":1}
W20260926 16:37:59.126096 21087 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:59.120033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:59745 (local address 127.11.158.190:39029)
0926 16:37:59.120245 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:59.120250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:59.120425 (+ 175us) server_negotiation.cc:407] Connection header received
0926 16:37:59.120579 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:59.120582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:59.120650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:59.120749 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:59.121207 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.121754 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.122587 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.122826 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.122934 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:59.125944 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:59745: BlockingRecv error: recv got EOF from 127.11.158.130:59745 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 26 16:37:59 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:59 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:59.311250 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:59.304450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:60193)
0926 16:37:59.304770 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:59.304785 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:59.304905 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:59.305210 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:59.305214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:59.305421 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:59.305526 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:59.305533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.306523 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.306529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:59.307376 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:59.307388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.308054 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.308057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:59.308142 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:59.308780 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:59.308793 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:59.311111 (+ 2318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":137,"threads_started":1}
W20260926 16:37:59.311462 21087 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:59.304583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:60193 (local address 127.11.158.190:39029)
0926 16:37:59.304733 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:59.304738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:59.304870 (+ 132us) server_negotiation.cc:407] Connection header received
0926 16:37:59.304981 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:59.304984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:59.305040 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:59.305130 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:59.305644 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.306340 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.307505 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.307924 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.308020 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:59.311304 (+ 3284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:60193: BlockingRecv error: recv got EOF from 127.11.158.131:60193 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 26 16:37:59 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:37:59 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:37:59.925292 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:37:59.919973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:53603)
0926 16:37:59.920230 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 16:37:59.920260 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 16:37:59.920371 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:37:59.920645 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:37:59.920647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:37:59.920822 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:37:59.920931 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:59.920937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.921673 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.921675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:59.922208 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:37:59.922215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.922620 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.922640 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:37:59.922698 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:59.923245 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:37:59.923256 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:37:59.925105 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
W20260926 16:37:59.925346 21091 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:37:59.920031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:53603 (local address 127.11.158.190:39029)
0926 16:37:59.920292 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 16:37:59.920296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:37:59.920309 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:37:59.920464 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:37:59.920467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:37:59.920512 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:37:59.920607 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:37:59.921037 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.921565 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.922311 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:37:59.922525 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:37:59.922619 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:37:59.925219 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:53603: BlockingRecv error: recv got EOF from 127.11.158.129:53603 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":61,"threads_started":1}
Sep 26 16:38:00 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:00 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:00.132771 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:00.127025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:58075)
0926 16:38:00.127289 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:00.127311 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:00.127413 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:00.127684 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:00.127687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:00.127910 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:00.128018 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:00.128025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.128896 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.128902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:00.129482 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:00.129490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.129983 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.129986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:00.130074 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:00.130640 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:00.130653 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:00.132651 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
W20260926 16:38:00.132963 21091 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:00.127129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:58075 (local address 127.11.158.190:39029)
0926 16:38:00.127287 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:00.127291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:00.127382 (+ 91us) server_negotiation.cc:407] Connection header received
0926 16:38:00.127512 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:00.127514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:00.127558 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:00.127639 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:00.128153 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.128781 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.129609 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.129861 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.129969 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:00.132804 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:58075: BlockingRecv error: recv got EOF from 127.11.158.130:58075 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 16:38:00 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:00 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:00.317637 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:00.312025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:50969)
0926 16:38:00.312515 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:00.312529 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:00.312676 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:00.312952 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:00.312955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:00.313164 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:00.313253 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:00.313259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.314010 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.314016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:00.314591 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:00.314599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.315048 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.315052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:00.315135 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:00.315684 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:00.315696 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:00.317506 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":100,"threads_started":1}
W20260926 16:38:00.317788 21091 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:00.312268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:50969 (local address 127.11.158.190:39029)
0926 16:38:00.312449 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:00.312465 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:00.312607 (+ 142us) server_negotiation.cc:407] Connection header received
0926 16:38:00.312779 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:00.312783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:00.312838 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:00.312926 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:00.313372 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.313896 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.314709 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.314933 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.315049 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:00.317635 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:50969: BlockingRecv error: recv got EOF from 127.11.158.131:50969 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 16:38:00 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:00 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:00.931464 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:00.926146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:49147)
0926 16:38:00.926411 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:00.926424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:00.926535 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:00.926873 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:00.926875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:00.927057 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:00.927152 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:00.927157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.927896 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.927899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:00.928514 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:00.928523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.928958 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.928961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:00.929034 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:00.929615 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:00.929627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:00.931349 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
W20260926 16:38:00.931632 21095 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:00.926205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:49147 (local address 127.11.158.190:39029)
0926 16:38:00.926431 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:00.926435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:00.926499 (+ 64us) server_negotiation.cc:407] Connection header received
0926 16:38:00.926640 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:00.926643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:00.926685 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:00.926764 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:00.927257 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.927793 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.928626 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:00.928840 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:00.928917 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:00.931483 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:49147: BlockingRecv error: recv got EOF from 127.11.158.129:49147 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":57,"threads_started":1}
Sep 26 16:38:01 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:01 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:01.139048 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:01.133556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:49255)
0926 16:38:01.134024 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:01.134039 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:01.134141 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:01.134413 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:01.134415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:01.134615 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:01.134714 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:01.134720 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.135422 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.135424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:01.135986 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:01.135994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.136463 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.136467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:01.136556 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:01.137033 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:01.137045 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:01.138913 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":92,"threads_started":1}
W20260926 16:38:01.139184 21095 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:01.133724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:49255 (local address 127.11.158.190:39029)
0926 16:38:01.133873 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:01.133876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:01.134098 (+ 222us) server_negotiation.cc:407] Connection header received
0926 16:38:01.134222 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:01.134224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:01.134269 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:01.134339 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:01.134818 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.135334 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.136085 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.136293 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.136425 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:01.139061 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:49255: BlockingRecv error: recv got EOF from 127.11.158.130:49255 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 16:38:01 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:01 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:01.324008 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:01.318429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:56927)
0926 16:38:01.318741 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:01.318754 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:01.318850 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:01.319110 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:01.319112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:01.319287 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:01.319378 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:01.319383 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.320087 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.320091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:01.320674 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:01.320682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.321115 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.321117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:01.321173 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:01.321716 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:01.321728 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:01.323825 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":84,"threads_started":1}
W20260926 16:38:01.324046 21095 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:01.318512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:56927 (local address 127.11.158.190:39029)
0926 16:38:01.318671 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:01.318674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:01.318822 (+ 148us) server_negotiation.cc:407] Connection header received
0926 16:38:01.318931 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:01.318933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:01.318994 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:01.319071 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:01.319476 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.319996 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.320793 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.321012 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.321102 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:01.323946 (+ 2844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:56927: BlockingRecv error: recv got EOF from 127.11.158.131:56927 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 16:38:01 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:01 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:01.938243 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:01.932515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:45979)
0926 16:38:01.932819 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:01.932832 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:01.932932 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:01.933164 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:01.933167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:01.933341 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:01.933434 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:01.933439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.934203 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.934206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:01.934816 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:01.934826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.935288 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.935291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:01.935357 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:01.935890 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:01.935902 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:01.938066 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":101,"threads_started":1}
W20260926 16:38:01.938376 21098 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:01.932545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:45979 (local address 127.11.158.190:39029)
0926 16:38:01.932792 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:01.932796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:01.932904 (+ 108us) server_negotiation.cc:407] Connection header received
0926 16:38:01.933001 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:01.933004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:01.933046 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:01.933129 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:01.933550 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.934075 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.934945 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:01.935179 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:01.935261 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:01.938218 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:45979: BlockingRecv error: recv got EOF from 127.11.158.129:45979 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
Sep 26 16:38:02 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:02 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:02.145411 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:02.139931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:48789)
0926 16:38:02.140259 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:02.140273 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:02.140420 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:02.140695 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:02.140698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:02.140880 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:02.141012 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:02.141018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.141738 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.141741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:02.142311 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:02.142319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.142743 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.142746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:02.142815 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:02.143346 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:02.143359 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:02.145296 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
W20260926 16:38:02.145614 21098 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:02.140148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:48789 (local address 127.11.158.190:39029)
0926 16:38:02.140297 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:02.140301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:02.140317 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:38:02.140513 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:02.140515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:02.140559 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:02.140632 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:02.141133 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.141640 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.142412 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.142620 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.142704 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:02.145451 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:48789: BlockingRecv error: recv got EOF from 127.11.158.130:48789 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 16:38:02 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:02 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:02.330757 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:02.324999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:60739)
0926 16:38:02.325278 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:02.325291 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:02.325426 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:02.325711 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:02.325713 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:02.325925 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:02.326022 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:02.326028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.326799 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.326805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:02.327374 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:02.327382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.327845 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.327849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:02.327936 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:02.328545 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:02.328557 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:02.330580 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
W20260926 16:38:02.330838 21098 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:02.325046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:60739 (local address 127.11.158.190:39029)
0926 16:38:02.325211 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:02.325217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:02.325367 (+ 150us) server_negotiation.cc:407] Connection header received
0926 16:38:02.325538 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:02.325541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:02.325595 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:02.325682 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:02.326149 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.326689 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.327499 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.327726 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.327833 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:02.330719 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:60739: BlockingRecv error: recv got EOF from 127.11.158.131:60739 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 16:38:02 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:02 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:02.952435 21102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:02.939240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:34365)
0926 16:38:02.939582 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:02.939606 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:02.939701 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:02.939944 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:02.939947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:02.940133 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:02.940227 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:02.940232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.941258 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.941262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:02.942094 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:02.942105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.942616 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.942619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:02.942700 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:02.950097 (+ 7397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:02.950113 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:02.952235 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":78,"threads_started":1}
W20260926 16:38:02.952651 21103 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:02.939331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:34365 (local address 127.11.158.190:39029)
0926 16:38:02.939680 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:02.939684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:02.939705 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:38:02.939772 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:02.939776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:02.939820 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:02.939906 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:02.940382 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.941121 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.942228 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:02.942496 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:02.942623 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:02.952465 (+ 9842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:34365: BlockingRecv error: recv got EOF from 127.11.158.129:34365 (error 108)
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":83,"threads_started":1}
Sep 26 16:38:03 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:03 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:03.151590 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:03.146193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:33937)
0926 16:38:03.146453 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:03.146466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:03.146590 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:03.146861 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:03.146864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:03.147046 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:03.147142 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:03.147147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.147868 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.147871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:03.148504 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:03.148514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.148947 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.148950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:03.149010 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:03.149507 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:03.149518 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:03.151444 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
W20260926 16:38:03.151736 21103 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:03.146296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:33937 (local address 127.11.158.190:39029)
0926 16:38:03.146438 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:03.146442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:03.146553 (+ 111us) server_negotiation.cc:407] Connection header received
0926 16:38:03.146671 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:03.146673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:03.146716 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:03.146784 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:03.147248 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.147766 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.148628 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.148851 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.148931 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:03.151604 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:33937: BlockingRecv error: recv got EOF from 127.11.158.130:33937 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 26 16:38:03 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:03 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.131@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:03.337532 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:03.331670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.131:40945)
0926 16:38:03.332040 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:03.332058 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:03.332225 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:03.332533 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:03.332536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:03.332745 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:03.332835 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:03.332841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.333613 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.333618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:03.334176 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:03.334184 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.334682 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.334685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:03.334791 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:03.335414 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:03.335430 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:03.337409 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":93,"threads_started":1}
W20260926 16:38:03.337864 21103 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:03.331817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:40945 (local address 127.11.158.190:39029)
0926 16:38:03.332019 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:03.332024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:03.332149 (+ 125us) server_negotiation.cc:407] Connection header received
0926 16:38:03.332364 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:03.332367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:03.332414 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:03.332508 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:03.332957 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.333495 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.334296 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.334578 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.334678 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:03.337590 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.131:40945: BlockingRecv error: recv got EOF from 127.11.158.131:40945 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 16:38:03 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:03 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.129@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:03.958648 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:03.953338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.129:42921)
0926 16:38:03.953624 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:03.953636 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:03.953718 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:03.953955 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:03.953957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:03.954139 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:03.954233 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:03.954238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.955014 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.955017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:03.955582 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:03.955591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.956083 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.956086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:03.956164 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:03.956724 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:03.956736 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:03.958529 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
W20260926 16:38:03.958801 21107 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:03.953487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:42921 (local address 127.11.158.190:39029)
0926 16:38:03.953740 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:03.953747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:03.953760 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:38:03.953796 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:03.953799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:03.953839 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:03.953923 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:03.954343 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.954897 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.955710 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:03.955945 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:03.956042 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:03.958663 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.129:42921: BlockingRecv error: recv got EOF from 127.11.158.129:42921 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
Sep 26 16:38:04 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:38:04 dist-test-slave-p39x krb5kdc[20048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.158.130@KRBTEST.COM for oryx/127.11.158.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:38:04.158864 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:04.152678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:39029 (local address 127.11.158.130:56087)
0926 16:38:04.152952 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:04.152966 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:04.153091 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:04.153352 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:04.153354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:04.153573 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:04.153662 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:04.153667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.154420 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.154426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:04.155015 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:04.155023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.155487 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.155490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:04.155569 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:04.156267 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:04.156279 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:04.158671 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.158.190:39029: Server oryx/127.11.158.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
W20260926 16:38:04.158939 21107 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:04.152755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:56087 (local address 127.11.158.190:39029)
0926 16:38:04.152904 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:04.152908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:04.153035 (+ 127us) server_negotiation.cc:407] Connection header received
0926 16:38:04.153188 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:04.153192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:04.153240 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:04.153329 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:04.153777 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.154312 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.155141 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.155377 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.155482 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:04.158819 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.158.130:56087: BlockingRecv error: recv got EOF from 127.11.158.130:56087 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:38:04.223097 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20631
I20260926 16:38:04.228637 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20783
I20260926 16:38:04.234141 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20916
I20260926 16:38:04.239444 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 20561
2026-09-26T16:38:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18050 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T16:38:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:38:04Z Disabled control of system clock
I20260926 16:38:04.270076 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:43735
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:43735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 16:38:04.384124 21112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:04.384464 21112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:04.384534 21112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:04.388651 21112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:38:04.388753 21112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:04.388782 21112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:04.388824 21112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:38:04.388859 21112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:38:04.393671 21112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:43735
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:43735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:04.394809 21112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:04.396198 21112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:04.406687 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:04.406839 21120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:04.406845 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:04.406929 21112 server_base.cc:1034] running on GCE node
I20260926 16:38:04.407816 21112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:04.409200 21112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:04.410415 21112 hybrid_clock.cc:648] HybridClock initialized: now 1790440684410400 us; error 33 us; skew 500 ppm
I20260926 16:38:04.412563 21112 webserver.cc:460] Webserver started at http://127.11.158.190:45391/ using document root <none> and password file <none>
I20260926 16:38:04.413209 21112 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:04.413318 21112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:04.413575 21112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:04.415843 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "6d6bf80cb3d548faa22b88ac96a73ea4"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "3e57470846daba741611207e65d6795a"
server_key_iv: "e0486dedbe31ea19fddb584b02ddff08"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.416522 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "6d6bf80cb3d548faa22b88ac96a73ea4"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "3e57470846daba741611207e65d6795a"
server_key_iv: "e0486dedbe31ea19fddb584b02ddff08"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.420719 21112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:38:04.423321 21126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:04.424750 21112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:38:04.424893 21112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "6d6bf80cb3d548faa22b88ac96a73ea4"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "3e57470846daba741611207e65d6795a"
server_key_iv: "e0486dedbe31ea19fddb584b02ddff08"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.425009 21112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:04.441490 21112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:04.442198 21112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:04.442399 21112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:04.449561 21112 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:43735
I20260926 16:38:04.449566 21178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:43735 every 8 connection(s)
I20260926 16:38:04.450508 21112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:38:04.453641 21179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:04.456223 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21112
I20260926 16:38:04.456368 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:38:04.456715 11898 external_mini_cluster.cc:1428] Setting key 147d6d226cf0905e3c3b0a544ffc5370
I20260926 16:38:04.459590 21179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4: Bootstrap starting.
I20260926 16:38:04.461740 21179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:04.462458 21179 log.cc:824] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:04.464290 21179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4: No bootstrap required, opened a new log
I20260926 16:38:04.466022 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:04.458157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.11.158.190:43735)
0926 16:38:04.458772 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:04.458786 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:04.458821 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:38:04.459390 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:04.459406 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:04.459855 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:04.460199 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:04.460639 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.461658 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.462602 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.463248 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.463413 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:04.464233 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:04.464238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:04.464256 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:04.464767 (+ 511us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:04.464821 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:04.464840 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:04.464948 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:04.465090 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 16:38:04.465344 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":128,"threads_started":1}
I20260926 16:38:04.467942 21179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43735 } }
I20260926 16:38:04.468237 21179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:04.468317 21179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6bf80cb3d548faa22b88ac96a73ea4, State: Initialized, Role: FOLLOWER
I20260926 16:38:04.468741 21179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43735 } }
I20260926 16:38:04.468888 21179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:38:04.468973 21179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:38:04.469069 21179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:04.469990 21179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43735 } }
I20260926 16:38:04.470357 21179 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6bf80cb3d548faa22b88ac96a73ea4; no voters:
I20260926 16:38:04.470702 21179 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:38:04.470810 21184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:38:04.471025 21184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [term 1 LEADER]: Becoming Leader. State: Replica: 6d6bf80cb3d548faa22b88ac96a73ea4, State: Running, Role: LEADER
I20260926 16:38:04.471395 21184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43735 } }
I20260926 16:38:04.471889 21179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:38:04.473156 21185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43735 } } }
I20260926 16:38:04.473170 21186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6bf80cb3d548faa22b88ac96a73ea4. Latest consensus state: current_term: 1 leader_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6bf80cb3d548faa22b88ac96a73ea4" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 43735 } } }
I20260926 16:38:04.473294 21185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:04.473294 21186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4 [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:04.473711 21193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:38:04.476761 21193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:38:04.481822 21193 catalog_manager.cc:1347] Generated new cluster ID: 2781e7eaa380416ebf74f2d4ef500dba
I20260926 16:38:04.481884 21193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:38:04.503053 21193 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:38:04.504438 21193 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:38:04.510435 21193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d6bf80cb3d548faa22b88ac96a73ea4: Generated new TSK 0
I20260926 16:38:04.511145 21193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:38:04.516117 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43735
--builtin_ntp_servers=127.11.158.148:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 16:38:04.628886 21203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:04.629191 21203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:04.629230 21203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:04.633273 21203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:04.633329 21203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:04.633384 21203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:38:04.638034 21203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:04.639055 21203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:04.640444 21203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:04.650039 21208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:04.650043 21211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:04.650039 21209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:04.650309 21203 server_base.cc:1034] running on GCE node
I20260926 16:38:04.650836 21203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:04.651407 21203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:04.652601 21203 hybrid_clock.cc:648] HybridClock initialized: now 1790440684652586 us; error 28 us; skew 500 ppm
I20260926 16:38:04.654318 21203 webserver.cc:460] Webserver started at http://127.11.158.129:37777/ using document root <none> and password file <none>
I20260926 16:38:04.654909 21203 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:04.654999 21203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:04.655227 21203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:04.657317 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "6d2e1aa8709443a6924a706073d8ffec"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "391e177a91bddb50c54e63677cb6e94d"
server_key_iv: "7c5143c36651084ed6ca0806d83f4c89"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.657869 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "6d2e1aa8709443a6924a706073d8ffec"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "391e177a91bddb50c54e63677cb6e94d"
server_key_iv: "7c5143c36651084ed6ca0806d83f4c89"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.661443 21203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260926 16:38:04.663954 21217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:04.665161 21203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:38:04.665290 21203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "6d2e1aa8709443a6924a706073d8ffec"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "391e177a91bddb50c54e63677cb6e94d"
server_key_iv: "7c5143c36651084ed6ca0806d83f4c89"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.665402 21203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:04.687896 21203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:04.688757 21203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:04.688968 21203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:04.689589 21203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:04.690567 21203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:04.690639 21203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:04.690711 21203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:04.690753 21203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:04.700165 21203 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:32971
I20260926 16:38:04.700201 21330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:32971 every 8 connection(s)
I20260926 16:38:04.701414 21203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:38:04.702445 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21203
I20260926 16:38:04.702584 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:38:04.703374 11898 external_mini_cluster.cc:1428] Setting key 13343d50bb97f17aef64494d569cc367
I20260926 16:38:04.706409 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43735
--builtin_ntp_servers=127.11.158.148:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 16:38:04.713658 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:04.704177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:46437 (local address 127.11.158.190:43735)
0926 16:38:04.704357 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:04.704364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:04.706188 (+ 1824us) server_negotiation.cc:407] Connection header received
0926 16:38:04.708064 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:04.708068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:04.708124 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:04.708207 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:04.709533 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.710095 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.711376 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.711623 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.711744 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:04.713102 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:04.713106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:04.713109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:04.713132 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:04.713153 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:04.713162 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:04.713241 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:04.713421 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 16:38:04.713502 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 16:38:04.714648 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:04.704930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:43735 (local address 127.11.158.129:46437)
0926 16:38:04.705973 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:04.706038 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:04.707693 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:04.708520 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:04.708533 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:04.708690 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:04.709330 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:04.709345 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.710269 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.710273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:04.711226 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:04.711237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.711798 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.711802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:04.711992 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:04.712743 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:38:04.712744 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:04.712791 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:38:04.712867 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:04.713257 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:04.713267 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:04.713273 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:38:04.713407 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 16:38:04.713731 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":297,"threads_started":1}
I20260926 16:38:04.716182 21331 heartbeater.cc:347] Connected to a master server at 127.11.158.190:43735
I20260926 16:38:04.716514 21331 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:04.717146 21331 heartbeater.cc:511] Master 127.11.158.190:43735 requested a full tablet report, sending...
I20260926 16:38:04.718945 21143 ts_manager.cc:194] Registered new tserver with Master: 6d2e1aa8709443a6924a706073d8ffec (127.11.158.129:32971)
I20260926 16:38:04.720634 21143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.129:46437
W20260926 16:38:04.825903 21334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:04.826187 21334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:04.826226 21334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:04.830343 21334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:04.830410 21334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:04.830466 21334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:38:04.835258 21334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:04.836402 21334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:04.837771 21334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:04.847432 21340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:04.847426 21339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:04.847488 21342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:04.847708 21334 server_base.cc:1034] running on GCE node
I20260926 16:38:04.848170 21334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:04.848826 21334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:04.849987 21334 hybrid_clock.cc:648] HybridClock initialized: now 1790440684849965 us; error 36 us; skew 500 ppm
I20260926 16:38:04.851807 21334 webserver.cc:460] Webserver started at http://127.11.158.130:40641/ using document root <none> and password file <none>
I20260926 16:38:04.852468 21334 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:04.852535 21334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:04.852747 21334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:04.854781 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "45a1a10970b840b69a9f138b19c23ebb"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "9eac0e7ae3a562f1e548547b87b60187"
server_key_iv: "8e0a654bb579b02be3a44708c465fdd0"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.855350 21334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "45a1a10970b840b69a9f138b19c23ebb"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "9eac0e7ae3a562f1e548547b87b60187"
server_key_iv: "8e0a654bb579b02be3a44708c465fdd0"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.859174 21334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:38:04.861892 21348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:04.863257 21334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:38:04.863412 21334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "45a1a10970b840b69a9f138b19c23ebb"
format_stamp: "Formatted at 2026-09-26 16:38:04 on dist-test-slave-p39x"
server_key: "9eac0e7ae3a562f1e548547b87b60187"
server_key_iv: "8e0a654bb579b02be3a44708c465fdd0"
server_key_version: "encryptionkey@0"
I20260926 16:38:04.863541 21334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:04.875571 21334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:04.876274 21334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:04.876504 21334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:04.877110 21334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:04.878067 21334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:04.878139 21334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:04.878212 21334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:04.878258 21334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:04.887311 21334 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:42209
I20260926 16:38:04.887337 21461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:42209 every 8 connection(s)
I20260926 16:38:04.888214 21334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:38:04.893949 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21334
I20260926 16:38:04.894049 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:38:04.894402 11898 external_mini_cluster.cc:1428] Setting key b4862450c98f48dbcf627e51ad9c2bad
I20260926 16:38:04.896415 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43735
--builtin_ntp_servers=127.11.158.148:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 16:38:04.897751 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:04.889982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:47269 (local address 127.11.158.190:43735)
0926 16:38:04.890123 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:04.890127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:04.891180 (+ 1053us) server_negotiation.cc:407] Connection header received
0926 16:38:04.892587 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:04.892590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:04.892635 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:04.892729 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:04.893604 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.894190 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.895640 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.895909 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.895989 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:04.897272 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:04.897275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:04.897277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:04.897287 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:04.897305 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:04.897313 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:04.897371 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:04.897568 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 16:38:04.897617 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:38:04.898648 21464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:04.890391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:43735 (local address 127.11.158.130:47269)
0926 16:38:04.891038 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:04.891077 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:04.892386 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:04.892891 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:04.892898 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:04.892979 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:04.893462 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:04.893474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.894319 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.894323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:04.895512 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:04.895526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:04.896081 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:04.896085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:04.896317 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:04.896958 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:38:04.896962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:04.897028 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:38:04.897114 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:04.897398 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:04.897409 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:04.897418 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:38:04.897548 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 16:38:04.897858 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260926 16:38:04.899725 21462 heartbeater.cc:347] Connected to a master server at 127.11.158.190:43735
I20260926 16:38:04.899979 21462 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:04.900565 21462 heartbeater.cc:511] Master 127.11.158.190:43735 requested a full tablet report, sending...
I20260926 16:38:04.901762 21143 ts_manager.cc:194] Registered new tserver with Master: 45a1a10970b840b69a9f138b19c23ebb (127.11.158.130:42209)
I20260926 16:38:04.902482 21143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.130:47269
W20260926 16:38:05.007893 21465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:05.008208 21465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:05.008278 21465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:05.012411 21465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:05.012496 21465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:05.012584 21465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:38:05.017262 21465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:05.018308 21465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:05.019697 21465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:05.029382 21470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.029402 21471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:05.029472 21465 server_base.cc:1034] running on GCE node
W20260926 16:38:05.029384 21473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:05.030093 21465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:05.030709 21465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:05.031867 21465 hybrid_clock.cc:648] HybridClock initialized: now 1790440685031847 us; error 31 us; skew 500 ppm
I20260926 16:38:05.033782 21465 webserver.cc:460] Webserver started at http://127.11.158.131:40629/ using document root <none> and password file <none>
I20260926 16:38:05.034359 21465 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:05.034421 21465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:05.034636 21465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:05.036717 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "26c15cfe30ce41a98f4b164083a018c6"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "58ed76b6029198e155977f3bf559a79c"
server_key_iv: "57828f54e24cc0ee61141745063ddfab"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.037261 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "26c15cfe30ce41a98f4b164083a018c6"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "58ed76b6029198e155977f3bf559a79c"
server_key_iv: "57828f54e24cc0ee61141745063ddfab"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.040858 21465 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:38:05.043324 21479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.044687 21465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:38:05.044785 21465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "26c15cfe30ce41a98f4b164083a018c6"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "58ed76b6029198e155977f3bf559a79c"
server_key_iv: "57828f54e24cc0ee61141745063ddfab"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.044869 21465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:05.060177 21465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:05.060865 21465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:05.061010 21465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:05.061568 21465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:05.062476 21465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:05.062523 21465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.062561 21465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:05.062577 21465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.071558 21465 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:44417
I20260926 16:38:05.071595 21592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:44417 every 8 connection(s)
I20260926 16:38:05.072474 21465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:38:05.073205 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21465
I20260926 16:38:05.073302 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:38:05.073588 11898 external_mini_cluster.cc:1428] Setting key 72c75c9c28bbb2cb7fbd5511df738db6
I20260926 16:38:05.081578 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.075928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:60815 (local address 127.11.158.190:43735)
0926 16:38:05.076043 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:05.076048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:05.076069 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:38:05.077227 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:05.077229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:05.077272 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:05.077338 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:05.078133 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:05.078694 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:05.079809 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:05.080008 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:05.080080 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:05.081193 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:05.081196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:05.081198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:05.081210 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:05.081228 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:05.081235 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:05.081296 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:05.081423 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 16:38:05.081471 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:38:05.082438 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.074964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:43735 (local address 127.11.158.131:60815)
0926 16:38:05.075670 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:05.075720 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:05.077055 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:05.077468 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:05.077474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:05.077553 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:05.078023 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:05.078036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:05.078798 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:05.078800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:05.079709 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:05.079718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:05.080125 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:05.080128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:05.080288 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:05.080888 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:38:05.080889 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:05.080927 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:38:05.080993 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:05.081309 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:05.081315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:05.081319 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:38:05.081407 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 16:38:05.081604 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":130,"threads_started":1}
I20260926 16:38:05.083536 21593 heartbeater.cc:347] Connected to a master server at 127.11.158.190:43735
I20260926 16:38:05.083753 21593 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:05.084254 21593 heartbeater.cc:511] Master 127.11.158.190:43735 requested a full tablet report, sending...
I20260926 16:38:05.085266 21143 ts_manager.cc:194] Registered new tserver with Master: 26c15cfe30ce41a98f4b164083a018c6 (127.11.158.131:44417)
I20260926 16:38:05.085932 21143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.131:60815
I20260926 16:38:05.087364 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 16:38:05.090256 21182 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:05.089296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.11.158.190:43735)
0926 16:38:05.089415 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:05.089419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:05.089587 (+ 168us) server_negotiation.cc:407] Connection header received
0926 16:38:05.089698 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:05.089700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:05.089739 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:05.089798 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:05.090132 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58152: BlockingRecv error: recv got EOF from 127.0.0.1:58152 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:38:05.090565 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21203
I20260926 16:38:05.096324 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21334
I20260926 16:38:05.102120 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21465
I20260926 16:38:05.107847 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21112
2026-09-26T16:38:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (866 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T16:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:38:05Z Disabled control of system clock
I20260926 16:38:05.137933 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40265
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:40265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 16:38:05.249442 21604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:05.249737 21604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:05.249775 21604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:05.253984 21604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:38:05.254050 21604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:05.254066 21604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:05.254084 21604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:38:05.254101 21604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:38:05.258955 21604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:40265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:05.259999 21604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:05.261447 21604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:38:05.270166 21604 server_base.cc:1034] running on GCE node
W20260926 16:38:05.270136 21609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.270116 21610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.270102 21612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:05.271168 21604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:05.272292 21604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:05.273504 21604 hybrid_clock.cc:648] HybridClock initialized: now 1790440685273475 us; error 46 us; skew 500 ppm
I20260926 16:38:05.275377 21604 webserver.cc:460] Webserver started at http://127.11.158.190:34781/ using document root <none> and password file <none>
I20260926 16:38:05.275956 21604 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:05.276017 21604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:05.276234 21604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:05.278343 21604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "b4d7086429604a6fb95e99dec1a91a6a"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "cf5f5cef9c0559c2c2f67e0ce402d6c2"
server_key_iv: "c9fb078e5b2f470b4e8a63e7a4fa8835"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.278925 21604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "b4d7086429604a6fb95e99dec1a91a6a"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "cf5f5cef9c0559c2c2f67e0ce402d6c2"
server_key_iv: "c9fb078e5b2f470b4e8a63e7a4fa8835"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.282687 21604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:38:05.285176 21618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.286412 21604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:38:05.286550 21604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "b4d7086429604a6fb95e99dec1a91a6a"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "cf5f5cef9c0559c2c2f67e0ce402d6c2"
server_key_iv: "c9fb078e5b2f470b4e8a63e7a4fa8835"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.286686 21604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:05.291365 21604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:05.291970 21604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:05.292155 21604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:05.299263 21604 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:40265
I20260926 16:38:05.299288 21670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:40265 every 8 connection(s)
I20260926 16:38:05.300220 21604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:38:05.303500 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21604
I20260926 16:38:05.303578 21671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:05.303653 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:38:05.303948 11898 external_mini_cluster.cc:1428] Setting key e57576c5b62f73e8e8dc5426ce28fce8
I20260926 16:38:05.309887 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.305437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.11.158.190:40265)
0926 16:38:05.306127 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:05.306141 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:05.306180 (+ 39us) server_negotiation.cc:407] Connection header received
0926 16:38:05.306889 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:05.306908 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:05.307284 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:05.307669 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:05.308623 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:05.308628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:05.308642 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:05.308653 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:05.308708 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:05.308723 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:05.308878 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:05.308947 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 16:38:05.309199 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":157,"threads_started":1}
I20260926 16:38:05.310667 21671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a: Bootstrap starting.
I20260926 16:38:05.313195 21671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:05.313923 21671 log.cc:824] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:05.315744 21671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a: No bootstrap required, opened a new log
I20260926 16:38:05.319653 21671 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d7086429604a6fb95e99dec1a91a6a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40265 } }
I20260926 16:38:05.319931 21671 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:05.319998 21671 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d7086429604a6fb95e99dec1a91a6a, State: Initialized, Role: FOLLOWER
I20260926 16:38:05.320489 21671 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d7086429604a6fb95e99dec1a91a6a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40265 } }
I20260926 16:38:05.320664 21671 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:38:05.320736 21671 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:38:05.320835 21671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:05.321926 21671 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d7086429604a6fb95e99dec1a91a6a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40265 } }
I20260926 16:38:05.322309 21671 leader_election.cc:302] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4d7086429604a6fb95e99dec1a91a6a; no voters:
I20260926 16:38:05.322650 21671 leader_election.cc:288] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:38:05.322775 21676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:38:05.323045 21676 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [term 1 LEADER]: Becoming Leader. State: Replica: b4d7086429604a6fb95e99dec1a91a6a, State: Running, Role: LEADER
I20260926 16:38:05.323349 21676 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d7086429604a6fb95e99dec1a91a6a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40265 } }
I20260926 16:38:05.323922 21671 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:38:05.325140 21677 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4d7086429604a6fb95e99dec1a91a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d7086429604a6fb95e99dec1a91a6a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40265 } } }
I20260926 16:38:05.325150 21678 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4d7086429604a6fb95e99dec1a91a6a. Latest consensus state: current_term: 1 leader_uuid: "b4d7086429604a6fb95e99dec1a91a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4d7086429604a6fb95e99dec1a91a6a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40265 } } }
I20260926 16:38:05.325284 21678 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:05.325284 21677 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:05.325732 21685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:38:05.328903 21685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:38:05.334570 21685 catalog_manager.cc:1347] Generated new cluster ID: d18e38c7b2cd4f4aa2625b18dd0c44b7
I20260926 16:38:05.334661 21685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:38:05.351526 21685 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:38:05.351881 21685 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:38:05.361240 21685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4d7086429604a6fb95e99dec1a91a6a: Generated new TSK 0
I20260926 16:38:05.361899 21685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:38:05.376778 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40265
--builtin_ntp_servers=127.11.158.148:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 16:38:05.490693 21695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:05.491027 21695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:05.491106 21695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:05.495342 21695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:05.495409 21695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:05.495532 21695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:38:05.500512 21695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:05.501820 21695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:05.503400 21695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:05.513746 21703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.513777 21700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:05.513844 21695 server_base.cc:1034] running on GCE node
W20260926 16:38:05.513790 21701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:05.514425 21695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:05.515092 21695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:05.516259 21695 hybrid_clock.cc:648] HybridClock initialized: now 1790440685516225 us; error 48 us; skew 500 ppm
I20260926 16:38:05.518136 21695 webserver.cc:460] Webserver started at http://127.11.158.129:39081/ using document root <none> and password file <none>
I20260926 16:38:05.518721 21695 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:05.518785 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:05.518994 21695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:05.521143 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "6ddf599fabae409a94edea197fe91a2a"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "7a1641d091c66228a32b417d0bc1b652"
server_key_iv: "e4b836d769a899314e63f795545b408c"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.521711 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "6ddf599fabae409a94edea197fe91a2a"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "7a1641d091c66228a32b417d0bc1b652"
server_key_iv: "e4b836d769a899314e63f795545b408c"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.525790 21695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:38:05.528445 21709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.529740 21695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:38:05.529875 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "6ddf599fabae409a94edea197fe91a2a"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "7a1641d091c66228a32b417d0bc1b652"
server_key_iv: "e4b836d769a899314e63f795545b408c"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.529996 21695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:05.534768 21695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:05.535378 21695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:05.535561 21695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:05.536108 21695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:05.537062 21695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:05.537137 21695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.537201 21695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:05.537251 21695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.546355 21695 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:42711
I20260926 16:38:05.546376 21822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:42711 every 8 connection(s)
I20260926 16:38:05.547310 21695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:38:05.553051 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21695
I20260926 16:38:05.553193 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:38:05.553524 11898 external_mini_cluster.cc:1428] Setting key 503c6bfabbec480289016b5721eb9c78
I20260926 16:38:05.553673 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.549269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:57797 (local address 127.11.158.190:40265)
0926 16:38:05.549419 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:05.549424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:05.550553 (+ 1129us) server_negotiation.cc:407] Connection header received
0926 16:38:05.551807 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:05.551809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:05.551846 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:05.551920 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:05.553106 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:05.553110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:05.553113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:05.553127 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:05.553155 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:05.553169 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:05.553270 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:05.553462 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 16:38:05.553510 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:38:05.554697 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.549733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40265 (local address 127.11.158.129:57797)
0926 16:38:05.550408 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:05.550448 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:05.551626 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:05.552085 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:05.552091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:05.552201 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:05.552844 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:38:05.552846 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:05.552886 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:38:05.552953 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:05.553318 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:05.553326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:05.553331 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 16:38:05.553471 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 16:38:05.553737 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":104,"threads_started":1}
I20260926 16:38:05.555604 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40265
--builtin_ntp_servers=127.11.158.148:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 16:38:05.555907 21823 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40265
I20260926 16:38:05.556144 21823 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:05.556666 21823 heartbeater.cc:511] Master 127.11.158.190:40265 requested a full tablet report, sending...
I20260926 16:38:05.558485 21635 ts_manager.cc:194] Registered new tserver with Master: 6ddf599fabae409a94edea197fe91a2a (127.11.158.129:42711)
W20260926 16:38:05.671617 21826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:05.671901 21826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:05.671942 21826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:05.676139 21826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:05.676213 21826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:05.676273 21826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:38:05.681288 21826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:05.682461 21826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:05.683827 21826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:05.694254 21834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.694242 21832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.694268 21831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:05.694620 21826 server_base.cc:1034] running on GCE node
I20260926 16:38:05.695147 21826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:05.695876 21826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:05.697073 21826 hybrid_clock.cc:648] HybridClock initialized: now 1790440685697055 us; error 38 us; skew 500 ppm
I20260926 16:38:05.699185 21826 webserver.cc:460] Webserver started at http://127.11.158.130:39275/ using document root <none> and password file <none>
I20260926 16:38:05.699810 21826 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:05.699905 21826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:05.700132 21826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:05.702636 21826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "fc392e78bd1c4d04bf24234621f73bfd"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "7a6974d1c7656edc64a2baa2c5f744be"
server_key_iv: "ae07025ef45f3e628f04bc54d51191fc"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.703491 21826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "fc392e78bd1c4d04bf24234621f73bfd"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "7a6974d1c7656edc64a2baa2c5f744be"
server_key_iv: "ae07025ef45f3e628f04bc54d51191fc"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.708201 21826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260926 16:38:05.711180 21840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.712710 21826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:38:05.712867 21826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "fc392e78bd1c4d04bf24234621f73bfd"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "7a6974d1c7656edc64a2baa2c5f744be"
server_key_iv: "ae07025ef45f3e628f04bc54d51191fc"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.713006 21826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:05.717818 21826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:05.718525 21826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:05.718715 21826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:05.719295 21826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:05.720237 21826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:05.720315 21826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.720424 21826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:05.720456 21826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.729771 21826 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:42597
I20260926 16:38:05.729831 21953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:42597 every 8 connection(s)
I20260926 16:38:05.730749 21826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:38:05.732663 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21826
I20260926 16:38:05.732784 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:38:05.733114 11898 external_mini_cluster.cc:1428] Setting key 50435efbed4f44f64e889088efdd6e94
I20260926 16:38:05.735090 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40265
--builtin_ntp_servers=127.11.158.148:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 16:38:05.737658 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.732628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:46649 (local address 127.11.158.190:40265)
0926 16:38:05.732800 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:05.732805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:05.734131 (+ 1326us) server_negotiation.cc:407] Connection header received
0926 16:38:05.735673 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:05.735680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:05.735726 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:05.735821 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:05.737174 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:05.737181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:05.737186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:05.737201 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:05.737230 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:05.737244 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:05.737352 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:05.737503 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 16:38:05.737551 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 16:38:05.738565 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.733226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40265 (local address 127.11.158.130:46649)
0926 16:38:05.733982 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:05.734035 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:05.735457 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:05.736032 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:05.736040 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:05.736132 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:05.736866 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:38:05.736868 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:05.736912 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:38:05.736995 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:05.737345 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:05.737352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:05.737366 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 16:38:05.737513 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 16:38:05.737740 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":133,"threads_started":1}
I20260926 16:38:05.739821 21954 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40265
I20260926 16:38:05.740064 21954 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:05.740593 21954 heartbeater.cc:511] Master 127.11.158.190:40265 requested a full tablet report, sending...
I20260926 16:38:05.741715 21635 ts_manager.cc:194] Registered new tserver with Master: fc392e78bd1c4d04bf24234621f73bfd (127.11.158.130:42597)
W20260926 16:38:05.851303 21957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:05.851630 21957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:05.851706 21957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:05.855913 21957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:05.856005 21957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:05.856101 21957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:38:05.861225 21957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:05.862368 21957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:05.863803 21957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:05.873479 21962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.873499 21963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:05.873499 21965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:05.873739 21957 server_base.cc:1034] running on GCE node
I20260926 16:38:05.874248 21957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:05.874858 21957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:05.876037 21957 hybrid_clock.cc:648] HybridClock initialized: now 1790440685876022 us; error 29 us; skew 500 ppm
I20260926 16:38:05.877996 21957 webserver.cc:460] Webserver started at http://127.11.158.131:44045/ using document root <none> and password file <none>
I20260926 16:38:05.878543 21957 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:05.878602 21957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:05.878769 21957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:05.880837 21957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "26efc86f386e450aa06b6a05fac84f0c"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "abc29d660943eec2eec93929bf068c77"
server_key_iv: "6d6cdb44740f9170ac758a89b036ccaa"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.881345 21957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "26efc86f386e450aa06b6a05fac84f0c"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "abc29d660943eec2eec93929bf068c77"
server_key_iv: "6d6cdb44740f9170ac758a89b036ccaa"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.885040 21957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:38:05.887568 21971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.888998 21957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:38:05.889106 21957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "26efc86f386e450aa06b6a05fac84f0c"
format_stamp: "Formatted at 2026-09-26 16:38:05 on dist-test-slave-p39x"
server_key: "abc29d660943eec2eec93929bf068c77"
server_key_iv: "6d6cdb44740f9170ac758a89b036ccaa"
server_key_version: "encryptionkey@0"
I20260926 16:38:05.889209 21957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:05.893910 21957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:05.894505 21957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:05.894662 21957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:05.895193 21957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:05.896090 21957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:05.896139 21957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.896181 21957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:05.896206 21957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:05.907322 21957 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:35329
I20260926 16:38:05.907343 22084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:35329 every 8 connection(s)
I20260926 16:38:05.908322 21957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:38:05.913940 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 21957
I20260926 16:38:05.914065 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:38:05.914361 11898 external_mini_cluster.cc:1428] Setting key 81e8b74c2369c4e8c4e31303952ca65d
I20260926 16:38:05.914958 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.910388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:33451 (local address 127.11.158.190:40265)
0926 16:38:05.910552 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:05.910557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:05.911695 (+ 1138us) server_negotiation.cc:407] Connection header received
0926 16:38:05.913072 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:05.913075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:05.913114 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:05.913187 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:05.914451 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:05.914457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:05.914460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:38:05.914473 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:05.914496 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:38:05.914508 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:05.914602 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:05.914765 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 16:38:05.914808 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 16:38:05.915974 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:05.910853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40265 (local address 127.11.158.131:33451)
0926 16:38:05.911528 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:05.911567 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:05.912851 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:05.913343 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:05.913349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:05.913436 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:05.914169 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:38:05.914171 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:05.914221 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:38:05.914289 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:05.914604 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:05.914611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:05.914615 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:38:05.914785 (+ 170us) client_negotiation.cc:240] Negotiation successful
0926 16:38:05.915047 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260926 16:38:05.917115 22085 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40265
I20260926 16:38:05.917371 22085 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:05.917788 22085 heartbeater.cc:511] Master 127.11.158.190:40265 requested a full tablet report, sending...
I20260926 16:38:05.918795 21635 ts_manager.cc:194] Registered new tserver with Master: 26efc86f386e450aa06b6a05fac84f0c (127.11.158.131:35329)
I20260926 16:38:05.928639 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 16:38:05.931742 21674 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:38:05.930908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.11.158.190:40265)
0926 16:38:05.931052 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:05.931057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:05.931072 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:38:05.931222 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:05.931225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:05.931260 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:05.931346 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:05.931601 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42172: BlockingRecv error: recv got EOF from 127.0.0.1:42172 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:38:05.931937 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21695
I20260926 16:38:05.937654 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21826
I20260926 16:38:05.942770 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21957
I20260926 16:38:05.948019 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 21604
2026-09-26T16:38:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (840 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:38:06 dist-test-slave-p39x krb5kdc[22095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:38:06 dist-test-slave-p39x krb5kdc[22095](info): set up 2 sockets
Sep 26 16:38:06 dist-test-slave-p39x krb5kdc[22095](info): commencing operation
krb5kdc: starting...
W20260926 16:38:07.009977 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:38:07 dist-test-slave-p39x krb5kdc[22095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:38:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:38:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:38:07.139837 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:35335
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:35335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:38:07.254400 22108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:07.254691 22108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:07.254734 22108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:07.258833 22108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:38:07.258908 22108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:07.258929 22108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:07.258946 22108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:38:07.258962 22108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:38:07.264706 22108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:35335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:35335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:07.265843 22108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:07.266865 22108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:07.275867 22113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:07.275946 22108 server_base.cc:1034] running on GCE node
W20260926 16:38:07.275848 22116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:07.275867 22114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:07.276857 22108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:07.278064 22108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:07.279273 22108 hybrid_clock.cc:648] HybridClock initialized: now 1790440687279250 us; error 36 us; skew 500 ppm
Sep 26 16:38:07 dist-test-slave-p39x krb5kdc[22095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440687, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:07.282200 22108 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:38:07.283090 22108 webserver.cc:460] Webserver started at http://127.11.158.190:40509/ using document root <none> and password file <none>
I20260926 16:38:07.283630 22108 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:07.283691 22108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:07.283857 22108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:07.286046 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "5076fac90c1042288fa632414387c3f5"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "245c75d0199eba886ffb0241e9f7988e"
server_key_iv: "40b30a35d29159a64150420a68473f15"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.286592 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "5076fac90c1042288fa632414387c3f5"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "245c75d0199eba886ffb0241e9f7988e"
server_key_iv: "40b30a35d29159a64150420a68473f15"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.290529 22108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 16:38:07.293264 22123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:07.294677 22108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:38:07.294833 22108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "5076fac90c1042288fa632414387c3f5"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "245c75d0199eba886ffb0241e9f7988e"
server_key_iv: "40b30a35d29159a64150420a68473f15"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.294960 22108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:07.308212 22108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:07.312254 22108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:07.312533 22108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:07.320739 22108 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:35335
I20260926 16:38:07.321065 22175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:35335 every 8 connection(s)
I20260926 16:38:07.321911 22108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:38:07.325271 22176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:07.326785 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 22108
I20260926 16:38:07.326915 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:38:07.327214 11898 external_mini_cluster.cc:1428] Setting key 0e765ffa33b490a245d1286bc3ddb2a4
I20260926 16:38:07.332267 22176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5: Bootstrap starting.
I20260926 16:38:07.335115 22176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:07.336139 22176 log.cc:824] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:38:07 dist-test-slave-p39x krb5kdc[22095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:38:07.338392 22176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5: No bootstrap required, opened a new log
I20260926 16:38:07.342058 22176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5076fac90c1042288fa632414387c3f5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 35335 } }
I20260926 16:38:07.342324 22176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:07.342406 22176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5076fac90c1042288fa632414387c3f5, State: Initialized, Role: FOLLOWER
I20260926 16:38:07.342538 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:07.328837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.11.158.190:35335)
0926 16:38:07.329296 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:07.329313 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:07.329351 (+ 38us) server_negotiation.cc:407] Connection header received
0926 16:38:07.330053 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:07.330072 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:07.330417 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:07.330715 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:07.331648 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.332764 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.333946 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.334445 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.334578 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:07.337632 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:07.337638 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:07.337653 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:07.337680 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:07.339483 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:07.339953 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:07.339966 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:07.339971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:07.340085 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:07.340413 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:07.340416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:07.340418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:07.340840 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:07.341071 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:07.341340 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 16:38:07.341652 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":157,"threads_started":1}
I20260926 16:38:07.342978 22176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5076fac90c1042288fa632414387c3f5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 35335 } }
I20260926 16:38:07.343142 22176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:38:07.343204 22176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:38:07.343302 22176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:07.344442 22176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5076fac90c1042288fa632414387c3f5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 35335 } }
I20260926 16:38:07.344820 22176 leader_election.cc:302] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5076fac90c1042288fa632414387c3f5; no voters:
I20260926 16:38:07.345368 22176 leader_election.cc:288] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:38:07.345486 22181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:38:07.345731 22181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [term 1 LEADER]: Becoming Leader. State: Replica: 5076fac90c1042288fa632414387c3f5, State: Running, Role: LEADER
I20260926 16:38:07.346050 22181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5076fac90c1042288fa632414387c3f5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 35335 } }
I20260926 16:38:07.346680 22176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:38:07.347738 22182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5076fac90c1042288fa632414387c3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5076fac90c1042288fa632414387c3f5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 35335 } } }
I20260926 16:38:07.347875 22182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:07.348173 22183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5076fac90c1042288fa632414387c3f5. Latest consensus state: current_term: 1 leader_uuid: "5076fac90c1042288fa632414387c3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5076fac90c1042288fa632414387c3f5" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 35335 } } }
I20260926 16:38:07.348266 22183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5 [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:07.348412 22190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:38:07.351593 22190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:38:07.357930 22190 catalog_manager.cc:1347] Generated new cluster ID: dd7f649271e2478dbe670f1ab2d2b92b
I20260926 16:38:07.358037 22190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:38:07.376348 22190 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:38:07.377760 22190 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:38:07.391285 22190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5: Generated new TSK 0
I20260926 16:38:07.392165 22190 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:38:07.433467 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:35335
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:38:07.557713 22202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:07.558274 22202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:07.558362 22202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:07.562934 22202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:07.563016 22202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:07.563118 22202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:38:07.569587 22202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:35335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:07.570843 22202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:07.571942 22202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:07.582000 22207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:07.582019 22202 server_base.cc:1034] running on GCE node
W20260926 16:38:07.582048 22208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:07.582223 22210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:07.582741 22202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:07.583395 22202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:07.584617 22202 hybrid_clock.cc:648] HybridClock initialized: now 1790440687584538 us; error 93 us; skew 500 ppm
Sep 26 16:38:07 dist-test-slave-p39x krb5kdc[22095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440687, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:07.588182 22202 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:38:07.589254 22202 webserver.cc:460] Webserver started at http://127.11.158.129:40051/ using document root <none> and password file <none>
I20260926 16:38:07.589888 22202 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:07.589985 22202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:07.590214 22202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:07.592453 22202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "2f7606af36014b98b657d3c1bc79e1e8"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "cb6ea39eb555aadf226607f2c4970a65"
server_key_iv: "37869db00cf21ed7b083d728e1681737"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.593084 22202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "2f7606af36014b98b657d3c1bc79e1e8"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "cb6ea39eb555aadf226607f2c4970a65"
server_key_iv: "37869db00cf21ed7b083d728e1681737"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.597769 22202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:38:07.600862 22217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:07.602357 22202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:38:07.602517 22202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "2f7606af36014b98b657d3c1bc79e1e8"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "cb6ea39eb555aadf226607f2c4970a65"
server_key_iv: "37869db00cf21ed7b083d728e1681737"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.602649 22202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:07.623976 22202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:07.627471 22202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:07.627696 22202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:07.628373 22202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:07.629266 22202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:07.629338 22202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:07.629413 22202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:07.629455 22202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:07.638660 22202 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:46121
I20260926 16:38:07.638687 22330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:46121 every 8 connection(s)
I20260926 16:38:07.639590 22202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:38:07.640923 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 22202
I20260926 16:38:07.641050 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:38:07.641386 11898 external_mini_cluster.cc:1428] Setting key e14489b49f7f80f5084c2dd8eebd204f
Sep 26 16:38:07 dist-test-slave-p39x krb5kdc[22095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440687, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:38:07.655078 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:07.641694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:44907 (local address 127.11.158.190:35335)
0926 16:38:07.641839 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:07.641849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:07.642886 (+ 1037us) server_negotiation.cc:407] Connection header received
0926 16:38:07.644323 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:07.644326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:07.644405 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:07.644494 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:07.645673 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.646306 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.647677 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.647928 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.648049 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:07.651029 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:07.651032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:07.651034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:07.651060 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:07.652656 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:07.653236 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:07.653239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:07.653241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:07.653346 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:07.653843 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:07.653846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:07.653847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:07.654062 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:07.654216 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:07.654753 (+ 537us) server_negotiation.cc:299] Negotiation successful
0926 16:38:07.654891 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:38:07.656075 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:07.642182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:35335 (local address 127.11.158.129:44907)
0926 16:38:07.642726 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:07.642786 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:07.644107 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:07.644665 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:07.644672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:07.645043 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:07.645526 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:07.645540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.646444 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.646447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:07.647553 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:07.647563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.648093 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.648098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:07.648310 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:07.649117 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:07.649135 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:07.650828 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:07.652833 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:07.652838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:07.652853 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:07.653140 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:07.653508 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:07.653512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:07.653514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:07.653719 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:07.654203 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:07.654211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:07.654490 (+ 279us) client_negotiation.cc:769] Sending connection context
0926 16:38:07.654701 (+ 211us) client_negotiation.cc:240] Negotiation successful
0926 16:38:07.655053 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":138,"threads_started":1}
I20260926 16:38:07.657502 22331 heartbeater.cc:347] Connected to a master server at 127.11.158.190:35335
I20260926 16:38:07.657785 22331 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:07.658311 22331 heartbeater.cc:511] Master 127.11.158.190:35335 requested a full tablet report, sending...
I20260926 16:38:07.660071 22140 ts_manager.cc:194] Registered new tserver with Master: 2f7606af36014b98b657d3c1bc79e1e8 (127.11.158.129:46121)
I20260926 16:38:07.661811 22140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:44907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:38:07.672595 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:35335
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:38:07.791204 22336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:07.791505 22336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:07.791553 22336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:07.795739 22336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:07.795825 22336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:07.795890 22336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:38:07.801710 22336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:35335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:07.802886 22336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:07.804205 22336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:07.815569 22344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:07.815584 22342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:07.815569 22341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:07.815836 22336 server_base.cc:1034] running on GCE node
I20260926 16:38:07.816457 22336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:07.817239 22336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:07.818457 22336 hybrid_clock.cc:648] HybridClock initialized: now 1790440687818388 us; error 62 us; skew 500 ppm
Sep 26 16:38:07 dist-test-slave-p39x krb5kdc[22095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440687, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:07.821486 22336 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:38:07.822376 22336 webserver.cc:460] Webserver started at http://127.11.158.130:38823/ using document root <none> and password file <none>
I20260926 16:38:07.822917 22336 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:07.822975 22336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:07.823146 22336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:07.825259 22336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "56b59a41eab449dda49eb08df46a8186"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "781c01d427918167a4a16b4faf23b8d8"
server_key_iv: "e59fe99d00c5270fc7ece84e7563017d"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.825781 22336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "56b59a41eab449dda49eb08df46a8186"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "781c01d427918167a4a16b4faf23b8d8"
server_key_iv: "e59fe99d00c5270fc7ece84e7563017d"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.829619 22336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260926 16:38:07.832082 22351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:07.833508 22336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:38:07.833608 22336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "56b59a41eab449dda49eb08df46a8186"
format_stamp: "Formatted at 2026-09-26 16:38:07 on dist-test-slave-p39x"
server_key: "781c01d427918167a4a16b4faf23b8d8"
server_key_iv: "e59fe99d00c5270fc7ece84e7563017d"
server_key_version: "encryptionkey@0"
I20260926 16:38:07.833701 22336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:07.857048 22336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:07.860462 22336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:07.860633 22336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:07.861172 22336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:07.861974 22336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:07.862020 22336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:07.862062 22336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:07.862078 22336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:07.872699 22336 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:38597
I20260926 16:38:07.872699 22464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:38597 every 8 connection(s)
I20260926 16:38:07.873745 22336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:38:07.879994 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 22336
I20260926 16:38:07.880125 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:38:07.880482 11898 external_mini_cluster.cc:1428] Setting key 52362bfe0dbbab4d8e8b4165850992f2
Sep 26 16:38:07 dist-test-slave-p39x krb5kdc[22095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440687, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:38:07.888767 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:07.875594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:52837 (local address 127.11.158.190:35335)
0926 16:38:07.875726 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:07.875730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:07.876614 (+ 884us) server_negotiation.cc:407] Connection header received
0926 16:38:07.877984 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:07.877988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:07.878038 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:07.878132 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:07.879331 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.880021 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.881558 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.881889 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.882021 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:07.884706 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:07.884710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:07.884712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:07.884737 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:07.886197 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:07.886887 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:07.886890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:07.886891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:07.886972 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:07.887424 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:07.887426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:07.887427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:07.887615 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:07.887690 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:07.888517 (+ 827us) server_negotiation.cc:299] Negotiation successful
0926 16:38:07.888637 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:38:07.890055 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:07.875924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:35335 (local address 127.11.158.130:52837)
0926 16:38:07.876455 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:07.876499 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:07.877774 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:07.878280 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:07.878287 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:07.878654 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:07.879150 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:07.879165 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.880202 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.880206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:07.881420 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:07.881431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:07.882063 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:07.882068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:07.882262 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:07.882900 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:07.882917 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:07.884512 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:07.886331 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:07.886340 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:07.886362 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:07.886769 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:07.887098 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:07.887104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:07.887109 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:07.887319 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:07.887727 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:07.887737 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:07.888133 (+ 396us) client_negotiation.cc:769] Sending connection context
0926 16:38:07.888416 (+ 283us) client_negotiation.cc:240] Negotiation successful
0926 16:38:07.888833 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:38:07.891517 22465 heartbeater.cc:347] Connected to a master server at 127.11.158.190:35335
I20260926 16:38:07.891791 22465 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:07.892385 22465 heartbeater.cc:511] Master 127.11.158.190:35335 requested a full tablet report, sending...
I20260926 16:38:07.893539 22140 ts_manager.cc:194] Registered new tserver with Master: 56b59a41eab449dda49eb08df46a8186 (127.11.158.130:38597)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:38:07.894466 22140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:52837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:38:07.918236 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:35335
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:38:08.038632 22470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:08.038975 22470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:08.039054 22470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:08.043313 22470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:08.043431 22470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:08.043535 22470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:38:08.049860 22470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:35335
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:08.051064 22470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:08.052201 22470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:08.062307 22478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:08.062299 22475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:08.062299 22476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:08.062798 22470 server_base.cc:1034] running on GCE node
I20260926 16:38:08.063216 22470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:08.063889 22470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:08.065080 22470 hybrid_clock.cc:648] HybridClock initialized: now 1790440688065053 us; error 39 us; skew 500 ppm
Sep 26 16:38:08 dist-test-slave-p39x krb5kdc[22095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:08.072086 22470 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:38:08.073330 22470 webserver.cc:460] Webserver started at http://127.11.158.131:33053/ using document root <none> and password file <none>
I20260926 16:38:08.073963 22470 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:08.074064 22470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:08.074307 22470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:38:08.076550 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "117138541891471fa8471030877755cc"
format_stamp: "Formatted at 2026-09-26 16:38:08 on dist-test-slave-p39x"
server_key: "43dc4b37664f6972112e08a37318b78f"
server_key_iv: "de5c1d13f22229ee278e2aa5beba7399"
server_key_version: "encryptionkey@0"
I20260926 16:38:08.077155 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "117138541891471fa8471030877755cc"
format_stamp: "Formatted at 2026-09-26 16:38:08 on dist-test-slave-p39x"
server_key: "43dc4b37664f6972112e08a37318b78f"
server_key_iv: "de5c1d13f22229ee278e2aa5beba7399"
server_key_version: "encryptionkey@0"
I20260926 16:38:08.081130 22470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:38:08.084388 22485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:08.086297 22470 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:38:08.086441 22470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "117138541891471fa8471030877755cc"
format_stamp: "Formatted at 2026-09-26 16:38:08 on dist-test-slave-p39x"
server_key: "43dc4b37664f6972112e08a37318b78f"
server_key_iv: "de5c1d13f22229ee278e2aa5beba7399"
server_key_version: "encryptionkey@0"
I20260926 16:38:08.086596 22470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:08.100925 22470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:08.104808 22470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:08.105036 22470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:08.105679 22470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:08.106758 22470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:08.106813 22470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:08.106855 22470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:08.106910 22470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:08.117313 22470 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:41055
I20260926 16:38:08.117357 22598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:41055 every 8 connection(s)
I20260926 16:38:08.118345 22470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:38:08.125733 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 22470
I20260926 16:38:08.125908 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:38:08.126245 11898 external_mini_cluster.cc:1428] Setting key 69f6611d4c6543583b04228959329da5
Sep 26 16:38:08 dist-test-slave-p39x krb5kdc[22095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440688, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:38:08.134447 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.120406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:40389 (local address 127.11.158.190:35335)
0926 16:38:08.120585 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.120590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.121719 (+ 1129us) server_negotiation.cc:407] Connection header received
0926 16:38:08.123300 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.123304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.123358 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.123441 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:08.124802 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.125348 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.126943 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.127267 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.127349 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.130605 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:08.130608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:08.130610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:08.130638 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:08.132208 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:08.132872 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:08.132875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:08.132877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:08.132961 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:08.133334 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:08.133336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:08.133338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:08.133545 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:08.133622 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.134202 (+ 580us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.134318 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 16:38:08.135356 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.120844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:35335 (local address 127.11.158.131:40389)
0926 16:38:08.121509 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:08.121582 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:08.123063 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:08.123638 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:08.123645 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:08.124022 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:08.124628 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.124653 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.125531 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.125536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.126827 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.126837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.127632 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.127639 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.127867 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.128711 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:08.128727 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:08.130407 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:08.132402 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:08.132416 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:08.132437 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:08.132748 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:08.133070 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:08.133073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:08.133075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:08.133232 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:08.133672 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:08.133678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:08.133951 (+ 273us) client_negotiation.cc:769] Sending connection context
0926 16:38:08.134120 (+ 169us) client_negotiation.cc:240] Negotiation successful
0926 16:38:08.134430 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":153,"threads_started":1}
I20260926 16:38:08.136641 22599 heartbeater.cc:347] Connected to a master server at 127.11.158.190:35335
I20260926 16:38:08.136941 22599 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:08.137499 22599 heartbeater.cc:511] Master 127.11.158.190:35335 requested a full tablet report, sending...
I20260926 16:38:08.138602 22140 ts_manager.cc:194] Registered new tserver with Master: 117138541891471fa8471030877755cc (127.11.158.131:41055)
I20260926 16:38:08.139268 22140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:40389
I20260926 16:38:08.141971 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:38:08.152418 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.144425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.11.158.190:35335)
0926 16:38:08.144567 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.144573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.144752 (+ 179us) server_negotiation.cc:407] Connection header received
0926 16:38:08.144910 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.144913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.144959 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.145071 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:08.145784 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.146334 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.147260 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.147609 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.147709 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.149207 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:08.149212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:08.149214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:08.149241 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:08.150576 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:08.151046 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:08.151050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:08.151052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:08.151143 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:08.151417 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:08.151421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:08.151423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:08.151665 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:08.151813 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.152078 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.152219 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:38:08.155575 22140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:38:08.158746 22140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:38:08.173607 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.167326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:46121 (local address 127.0.0.1:35602)
0926 16:38:08.167955 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:08.168010 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:08.168140 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:08.169218 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:08.169224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:08.169245 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:08.169381 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.169388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.170787 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.170791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.171885 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.171895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.173058 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.173062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.173168 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.173204 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 16:38:08.173320 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 16:38:08.173439 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":16,"thread_start_us":86,"threads_started":1}
I20260926 16:38:08.173619 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.167521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.11.158.129:46121)
0926 16:38:08.168773 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.168789 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.168808 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:38:08.168870 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.168874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.169071 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.169227 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:08.169522 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.170660 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.172019 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.172907 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.173058 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.173218 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.173347 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.173445 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":104,"threads_started":1}
I20260926 16:38:08.173919 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.168534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.11.158.130:38597)
0926 16:38:08.168972 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.168984 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.169007 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:38:08.169078 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.169083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.169230 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.169362 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:08.169835 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.171062 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.172404 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.172966 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.173119 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.173238 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.173680 (+ 442us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.173783 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":134,"threads_started":1}
I20260926 16:38:08.173614 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.168094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38597 (local address 127.0.0.1:57628)
0926 16:38:08.168550 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:08.168564 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:08.168660 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:08.169531 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:08.169534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:08.169575 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:08.169691 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.169698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.171194 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.171201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.171993 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.172004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.173300 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.173306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.173387 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.173406 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:38:08.173468 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 16:38:08.173525 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":86,"threads_started":1}
I20260926 16:38:08.175170 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.167564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:41055 (local address 127.0.0.1:42566)
0926 16:38:08.168452 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:08.168466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:08.168557 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:08.169377 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:08.169380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:08.169394 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:08.169497 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.169504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.170588 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.170592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.173046 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.173057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.174725 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.174729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.174842 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.174868 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:38:08.174948 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:38:08.175022 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":34,"thread_start_us":95,"threads_started":1}
I20260926 16:38:08.175509 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.168114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.11.158.131:41055)
0926 16:38:08.168811 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.168827 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.168852 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:38:08.168931 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.168937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.169157 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.169315 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:08.169617 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.170442 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.173194 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.174606 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.174742 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.175028 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.175080 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.175164 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":121,"threads_started":1}
I20260926 16:38:08.176924 22265 tablet_service.cc:1467] Processing CreateTablet for tablet cf2fb2eab9074a8bb432322c66b48658 (DEFAULT_TABLE table=test-table [id=221ba6c942b2496a8f1c600b285054c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:38:08.177870 22265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2fb2eab9074a8bb432322c66b48658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:08.177510 22399 tablet_service.cc:1467] Processing CreateTablet for tablet cf2fb2eab9074a8bb432322c66b48658 (DEFAULT_TABLE table=test-table [id=221ba6c942b2496a8f1c600b285054c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:38:08.178545 22399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2fb2eab9074a8bb432322c66b48658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:08.178169 22533 tablet_service.cc:1467] Processing CreateTablet for tablet cf2fb2eab9074a8bb432322c66b48658 (DEFAULT_TABLE table=test-table [id=221ba6c942b2496a8f1c600b285054c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:38:08.179104 22533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2fb2eab9074a8bb432322c66b48658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:08.184495 22616 tablet_bootstrap.cc:492] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8: Bootstrap starting.
I20260926 16:38:08.186229 22618 tablet_bootstrap.cc:492] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Bootstrap starting.
I20260926 16:38:08.186573 22617 tablet_bootstrap.cc:492] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Bootstrap starting.
I20260926 16:38:08.186833 22616 tablet_bootstrap.cc:654] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:08.187528 22616 log.cc:824] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:08.188453 22618 tablet_bootstrap.cc:654] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:08.189039 22617 tablet_bootstrap.cc:654] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:08.189592 22616 tablet_bootstrap.cc:492] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8: No bootstrap required, opened a new log
I20260926 16:38:08.189752 22617 log.cc:824] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:08.189745 22618 log.cc:824] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:08.189793 22616 ts_tablet_manager.cc:1397] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:38:08.191787 22617 tablet_bootstrap.cc:492] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: No bootstrap required, opened a new log
I20260926 16:38:08.191787 22618 tablet_bootstrap.cc:492] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: No bootstrap required, opened a new log
I20260926 16:38:08.191970 22618 ts_tablet_manager.cc:1397] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260926 16:38:08.191970 22617 ts_tablet_manager.cc:1397] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260926 16:38:08.193552 22616 raft_consensus.cc:348] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.193783 22616 raft_consensus.cc:374] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:08.193835 22616 raft_consensus.cc:729] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f7606af36014b98b657d3c1bc79e1e8, State: Initialized, Role: FOLLOWER
I20260926 16:38:08.194294 22616 consensus_queue.cc:260] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.194929 22331 heartbeater.cc:503] Master 127.11.158.190:35335 was elected leader, sending a full tablet report...
I20260926 16:38:08.195294 22616 ts_tablet_manager.cc:1428] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:38:08.195951 22618 raft_consensus.cc:348] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.195945 22617 raft_consensus.cc:348] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.196204 22618 raft_consensus.cc:374] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:08.196204 22617 raft_consensus.cc:374] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:08.196259 22618 raft_consensus.cc:729] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117138541891471fa8471030877755cc, State: Initialized, Role: FOLLOWER
I20260926 16:38:08.196259 22617 raft_consensus.cc:729] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b59a41eab449dda49eb08df46a8186, State: Initialized, Role: FOLLOWER
I20260926 16:38:08.196730 22618 consensus_queue.cc:260] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.196730 22617 consensus_queue.cc:260] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.197635 22599 heartbeater.cc:503] Master 127.11.158.190:35335 was elected leader, sending a full tablet report...
I20260926 16:38:08.197741 22618 ts_tablet_manager.cc:1428] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260926 16:38:08.197741 22617 ts_tablet_manager.cc:1428] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:38:08.198066 22465 heartbeater.cc:503] Master 127.11.158.190:35335 was elected leader, sending a full tablet report...
I20260926 16:38:08.271481 22623 raft_consensus.cc:482] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:38:08.271734 22623 raft_consensus.cc:504] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.273020 22623 leader_election.cc:288] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f7606af36014b98b657d3c1bc79e1e8 (127.11.158.129:46121), 56b59a41eab449dda49eb08df46a8186 (127.11.158.130:38597)
I20260926 16:38:08.278039 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.273478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:41575 (local address 127.11.158.129:46121)
0926 16:38:08.273652 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.273658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.273708 (+ 50us) server_negotiation.cc:407] Connection header received
0926 16:38:08.273771 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.273774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.273829 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.273909 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:08.274241 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.275044 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.276652 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.277301 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.277414 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.277467 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.277840 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.277921 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260926 16:38:08.278039 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.273357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:46121 (local address 127.11.158.131:41575)
0926 16:38:08.273487 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:08.273504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:08.273619 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:08.273950 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:08.273953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:08.273974 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:08.274121 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.274130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.275178 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.275181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.276489 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.276501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.277534 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.277538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.277690 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.277746 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 16:38:08.277822 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 16:38:08.277899 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":34}
I20260926 16:38:08.278287 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.273778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:45547 (local address 127.11.158.130:38597)
0926 16:38:08.273921 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.273925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.274061 (+ 136us) server_negotiation.cc:407] Connection header received
0926 16:38:08.274163 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.274166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.274245 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.274325 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:08.274543 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.275255 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.276670 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.277285 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.277414 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.277467 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.278032 (+ 565us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.278135 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:38:08.278631 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.273628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38597 (local address 127.11.158.131:45547)
0926 16:38:08.273965 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:08.273977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:08.274067 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:08.274340 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:08.274342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:08.274355 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:08.274442 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.274447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.275359 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.275362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.276557 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:08.276566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.277479 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.277487 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:08.277765 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.277830 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 16:38:08.278422 (+ 592us) client_negotiation.cc:240] Negotiation successful
0926 16:38:08.278502 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":24,"thread_start_us":146,"threads_started":1}
I20260926 16:38:08.278784 22285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2fb2eab9074a8bb432322c66b48658" candidate_uuid: "117138541891471fa8471030877755cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f7606af36014b98b657d3c1bc79e1e8" is_pre_election: true
I20260926 16:38:08.279129 22285 raft_consensus.cc:2454] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117138541891471fa8471030877755cc in term 0.
I20260926 16:38:08.279292 22419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2fb2eab9074a8bb432322c66b48658" candidate_uuid: "117138541891471fa8471030877755cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b59a41eab449dda49eb08df46a8186" is_pre_election: true
I20260926 16:38:08.279598 22419 raft_consensus.cc:2454] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117138541891471fa8471030877755cc in term 0.
I20260926 16:38:08.279687 22488 leader_election.cc:302] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117138541891471fa8471030877755cc, 2f7606af36014b98b657d3c1bc79e1e8; no voters:
I20260926 16:38:08.279995 22623 raft_consensus.cc:2781] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:38:08.280085 22623 raft_consensus.cc:482] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:38:08.280143 22623 raft_consensus.cc:3037] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:08.281235 22623 raft_consensus.cc:504] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.281579 22623 leader_election.cc:288] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [CANDIDATE]: Term 1 election: Requested vote from peers 2f7606af36014b98b657d3c1bc79e1e8 (127.11.158.129:46121), 56b59a41eab449dda49eb08df46a8186 (127.11.158.130:38597)
I20260926 16:38:08.281994 22285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2fb2eab9074a8bb432322c66b48658" candidate_uuid: "117138541891471fa8471030877755cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f7606af36014b98b657d3c1bc79e1e8"
I20260926 16:38:08.282016 22419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2fb2eab9074a8bb432322c66b48658" candidate_uuid: "117138541891471fa8471030877755cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b59a41eab449dda49eb08df46a8186"
I20260926 16:38:08.282133 22285 raft_consensus.cc:3037] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:08.282141 22419 raft_consensus.cc:3037] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:08.283438 22285 raft_consensus.cc:2454] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117138541891471fa8471030877755cc in term 1.
I20260926 16:38:08.283438 22419 raft_consensus.cc:2454] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117138541891471fa8471030877755cc in term 1.
I20260926 16:38:08.283710 22488 leader_election.cc:302] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117138541891471fa8471030877755cc, 56b59a41eab449dda49eb08df46a8186; no voters:
I20260926 16:38:08.283886 22623 raft_consensus.cc:2781] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:38:08.284121 22623 raft_consensus.cc:686] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 1 LEADER]: Becoming Leader. State: Replica: 117138541891471fa8471030877755cc, State: Running, Role: LEADER
I20260926 16:38:08.284525 22623 consensus_queue.cc:237] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } }
I20260926 16:38:08.287969 22138 catalog_manager.cc:5495] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 117138541891471fa8471030877755cc (127.11.158.131). New cstate: current_term: 1 leader_uuid: "117138541891471fa8471030877755cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117138541891471fa8471030877755cc" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 41055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 } health_report { overall_health: UNKNOWN } } }
I20260926 16:38:08.328555 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.324283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.11.158.131:41055)
0926 16:38:08.324486 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:08.324491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:08.324509 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:38:08.324666 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:08.324668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:08.324731 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:08.324853 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:38:08.325132 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.325721 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.326686 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:08.326928 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:08.327052 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:08.327393 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:38:08.327914 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:38:08.328054 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:08.328254 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 16:38:08.328326 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260926 16:38:08.335654 22419 raft_consensus.cc:1261] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 1 FOLLOWER]: Refusing update from remote peer 117138541891471fa8471030877755cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:38:08.335661 22285 raft_consensus.cc:1261] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 1 FOLLOWER]: Refusing update from remote peer 117138541891471fa8471030877755cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:38:08.336474 22623 consensus_queue.cc:1035] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f7606af36014b98b657d3c1bc79e1e8" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:38:08.336648 22626 consensus_queue.cc:1035] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b59a41eab449dda49eb08df46a8186" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:38:08.345804 22631 mvcc.cc:205] Tried to move back new op lower bound from 7333645059414986752 to 7333645059216310272. Current Snapshot: MvccSnapshot[applied={T|T < 7333645059414986752}]
I20260926 16:38:08.354988 22138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:38:08.355172 22138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:38:08.357796 22138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5076fac90c1042288fa632414387c3f5: Sending DeleteTablet for 3 replicas of tablet cf2fb2eab9074a8bb432322c66b48658
I20260926 16:38:08.358701 22533 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2fb2eab9074a8bb432322c66b48658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:38:08 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:42566
I20260926 16:38:08.358896 22399 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2fb2eab9074a8bb432322c66b48658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:38:08 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:57628
I20260926 16:38:08.358901 22265 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2fb2eab9074a8bb432322c66b48658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:38:08 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:35602
I20260926 16:38:08.359403 22640 tablet_replica.cc:330] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8: stopping tablet replica
I20260926 16:38:08.359436 22641 tablet_replica.cc:330] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: stopping tablet replica
I20260926 16:38:08.359776 22640 raft_consensus.cc:2229] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:38:08.359835 22641 raft_consensus.cc:2229] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:38:08.359957 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 22202
I20260926 16:38:08.360146 22640 raft_consensus.cc:2258] T cf2fb2eab9074a8bb432322c66b48658 P 2f7606af36014b98b657d3c1bc79e1e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:38:08.360380 22641 raft_consensus.cc:2258] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:38:08.362316 22641 ts_tablet_manager.cc:1905] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:38:08.362617 22639 tablet_replica.cc:330] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: stopping tablet replica
I20260926 16:38:08.362967 22639 raft_consensus.cc:2229] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:38:08.363453 22639 raft_consensus.cc:2258] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:38:08.365008 22639 ts_tablet_manager.cc:1905] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:38:08.365952 22641 ts_tablet_manager.cc:1918] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:38:08.366061 22641 log.cc:1196] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/cf2fb2eab9074a8bb432322c66b48658
I20260926 16:38:08.366384 22641 ts_tablet_manager.cc:1939] T cf2fb2eab9074a8bb432322c66b48658 P 56b59a41eab449dda49eb08df46a8186: Deleting consensus metadata
I20260926 16:38:08.367851 22639 ts_tablet_manager.cc:1918] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:38:08.367851 22126 catalog_manager.cc:4841] TS 56b59a41eab449dda49eb08df46a8186 (127.11.158.130:38597): tablet cf2fb2eab9074a8bb432322c66b48658 (table test-table [id=221ba6c942b2496a8f1c600b285054c5]) successfully deleted
I20260926 16:38:08.368047 22639 log.cc:1196] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/cf2fb2eab9074a8bb432322c66b48658
I20260926 16:38:08.368426 22639 ts_tablet_manager.cc:1939] T cf2fb2eab9074a8bb432322c66b48658 P 117138541891471fa8471030877755cc: Deleting consensus metadata
W20260926 16:38:08.368906 22126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:46121 (error 108)
I20260926 16:38:08.369067 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 22336
I20260926 16:38:08.369654 22124 catalog_manager.cc:4841] TS 117138541891471fa8471030877755cc (127.11.158.131:41055): tablet cf2fb2eab9074a8bb432322c66b48658 (table test-table [id=221ba6c942b2496a8f1c600b285054c5]) successfully deleted
I20260926 16:38:08.371709 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:08.371435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:46121 (local address 127.0.0.1:35618)
0926 16:38:08.371570 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:08.371636 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 16:38:08.371910 22126 catalog_manager.cc:4584] TS 2f7606af36014b98b657d3c1bc79e1e8 (127.11.158.129:46121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf2fb2eab9074a8bb432322c66b48658: Network error: Client connection negotiation failed: client connection to 127.11.158.129:46121: connect: Connection refused (error 111)
I20260926 16:38:08.376274 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 22470
I20260926 16:38:08.381541 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 22108
2026-09-26T16:38:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2434 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:38:08 dist-test-slave-p39x krb5kdc[22645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:38:08 dist-test-slave-p39x krb5kdc[22645](info): set up 2 sockets
Sep 26 16:38:08 dist-test-slave-p39x krb5kdc[22645](info): commencing operation
krb5kdc: starting...
W20260926 16:38:09.433941 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:38:09 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:38:09Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/rangeradmin_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/rangeradmin_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/rangerlookup_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/rangerlookup_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
I20260926 16:38:09.620268 11898 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskupRbDL/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/postgres -l logfile start
2026-09-26 16:38:12.227 UTC [22675] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 16:38:12.227 UTC [22675] LOG: listening on IPv4 address "127.11.158.148", port 40601
2026-09-26 16:38:12.232 UTC [22675] LOG: listening on Unix socket "/tmp/.s.PGSQL.40601"
2026-09-26 16:38:12.255 UTC [22678] LOG: database system was shut down at 2026-09-26 16:38:10 UTC
2026-09-26 16:38:12.259 UTC [22675] LOG: database system is ready to accept connections
I20260926 16:38:13.233867 11898 mini_postgres.cc:96] Postgres bound to 40601
2026-09-26 16:38:13.240 UTC [22686] FATAL: database "slave" does not exist
127.11.158.148:40601 - accepting connections
I20260926 16:38:13.240936 11898 mini_ranger.cc:161] Starting Ranger...
I20260926 16:38:13.256134 11898 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 16:38:13.768930 11898 mini_ranger.cc:87] Created ranger Postgres database
I20260926 16:38:13.769029 11898 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-admin
2026-09-26 16:38:14,060 [I] DB FLAVOR :POSTGRES
2026-09-26 16:38:14,061 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:38:14,061 [I] Checking connection..
2026-09-26 16:38:14,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 16:38:14,303 [I] Checking connection passed.
2026-09-26 16:38:14,303 [I] --------- Verifying version history table ---------
2026-09-26 16:38:14,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:38:14,572 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:38:14,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:38:14,818 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:38:14,818 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 16:38:14,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 16:38:15.077 UTC [22750] WARNING: there is no transaction in progress
2026-09-26 16:38:15,083 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 16:38:15,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:38:15,333 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 16:38:15,333 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:38:15,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 16:38:15,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 16:38:15,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p39x.c.gcp-upstream.internal','N') ;"
2026-09-26 16:38:16,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:38:16,319 [I] Table x_portal_user does not exist in database ranger
2026-09-26 16:38:16,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:38:16,552 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 16:38:16,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:38:16,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:38:17,037 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 16:38:17,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 16:38:18.039 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.055 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.073 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.090 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.107 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.126 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.189 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.195 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.209 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.219 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.229 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.244 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.254 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.261 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.267 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18.274 UTC [22858] WARNING: there is no transaction in progress
2026-09-26 16:38:18,972 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 16:38:18,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 16:38:19,226 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 16:38:19,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:38:19,465 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 16:38:19,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:38:19,700 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 16:38:19,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:38:19,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:38:20,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p39x.c.gcp-upstream.internal';"
2026-09-26 16:38:20,467 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 16:38:20,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 16:38:20,712 [I] DB_PATCHES have already been applied
I20260926 16:38:20.717937 11898 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-admin:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskupRbDL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 16:38:20.718006 11898 mini_ranger.cc:193] Using host: 127.11.158.148
I20260926 16:38:20.721484 11898 mini_ranger.cc:227] Ranger admin URL: http://127.11.158.148:59363
Sep 26, 2026 4:38:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:38:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:38:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 4:38:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 4:38:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 4:38:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 4:38:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 16:38:21.833128 11898 mini_ranger.cc:160] Time spent starting Ranger: real 8.592s user 0.001s sys 0.009s
Sep 26, 2026 4:38:41 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 4:38:41 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 4:38:41 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 4:38:41 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 4:38:42 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 16:38:42.967479 11898 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/rangerkms_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/rangerkms_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.158.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.158.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 16:38:43 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440723, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 16:38:43.074841 11898 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 16:38:43.085621 11898 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 16:38:44.007640 11898 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 16:38:44.007764 11898 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms
I20260926 16:38:44.007871 11898 mini_ranger_kms.cc:227] Using postgres at 127.11.158.148:40601
2026-09-26 16:38:44,187 [I] DB FLAVOR :POSTGRES
2026-09-26 16:38:44,187 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:38:44,187 [I] Checking connection
2026-09-26 16:38:44,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 16:38:44,430 [I] connection success
2026-09-26 16:38:44,431 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 16:38:44,431 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 16:38:44,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 16:38:44,664 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 16:38:44,665 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:38:44,665 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 16:38:44,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:40601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 16:38:44,926 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 16:38:45.372737 11898 mini_ranger_kms.cc:310] Created kms service
I20260926 16:38:45.600458 11898 mini_ranger_kms.cc:326] Created kudu user
I20260926 16:38:45.644595 11898 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 16:38:45 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440723, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:38:46.046254 11898 mini_ranger_kms.cc:384] Added ranger policy
I20260926 16:38:46.046415 11898 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms:/tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 16:38:46.046437 11898 mini_ranger_kms.cc:240] Using host: 127.11.158.148
I20260926 16:38:46.051110 11898 mini_ranger_kms.cc:276] Ranger KMS PID: 23038
I20260926 16:38:46.051259 11898 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.158.148:45713
Sep 26, 2026 4:38:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:38:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:38:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 4:38:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 4:38:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 4:38:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 16:38:48.289825 11898 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.215s user 0.000s sys 0.165s
I20260926 16:38:48.289939 11898 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 16:38:48.290081 11898 mini_ranger_kms.cc:401] 127.11.158.148:45713/kms/v1/keys
Sep 26 16:38:48 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440728, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.158.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 16:38:48 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440728, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.158.148@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
Sep 26 16:38:49 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440728, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.158.148@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
Sep 26 16:38:51 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:38:51.479099 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38391
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38391
--ranger_config_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:45713/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:38:51.592113 23089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:51.592439 23089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:51.592510 23089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:51.596645 23089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:38:51.596704 23089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:51.596724 23089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:51.596769 23089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:38:51.596792 23089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:38:51.602931 23089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.158.148:45713/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38391
--ranger_config_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:51.604053 23089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:51.605176 23089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:51.614295 23094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:51.614478 23089 server_base.cc:1034] running on GCE node
W20260926 16:38:51.614632 23097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:51.614372 23095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:51.615250 23089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:51.616454 23089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:51.617635 23089 hybrid_clock.cc:648] HybridClock initialized: now 1790440731617598 us; error 55 us; skew 500 ppm
Sep 26 16:38:51 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440731, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:51.626742 23089 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:38:51.627804 23089 webserver.cc:460] Webserver started at http://127.11.158.190:33649/ using document root <none> and password file <none>
I20260926 16:38:51.628450 23089 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:51.628535 23089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:51.628787 23089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:38:51 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440731, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:38:51.676116 23089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "8a600dd81fba480c89a365387f129ade"
format_stamp: "Formatted at 2026-09-26 16:38:51 on dist-test-slave-p39x"
server_key: "241005002e66e714a1cd9ff349576c4e"
server_key_iv: "4151d3cd36c6fd63f3ee223c9a84f7f6"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:51.676815 23089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "8a600dd81fba480c89a365387f129ade"
format_stamp: "Formatted at 2026-09-26 16:38:51 on dist-test-slave-p39x"
server_key: "241005002e66e714a1cd9ff349576c4e"
server_key_iv: "4151d3cd36c6fd63f3ee223c9a84f7f6"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:51.680763 23089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:38:51.694832 23104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:51.696503 23089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:38:51.696661 23089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "8a600dd81fba480c89a365387f129ade"
format_stamp: "Formatted at 2026-09-26 16:38:51 on dist-test-slave-p39x"
server_key: "241005002e66e714a1cd9ff349576c4e"
server_key_iv: "4151d3cd36c6fd63f3ee223c9a84f7f6"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:51.696820 23089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:51.709982 23089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:51.712958 23089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:51.713173 23089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:51.721215 23089 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:38391
I20260926 16:38:51.721257 23156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:38391 every 8 connection(s)
I20260926 16:38:51.722205 23089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:38:51.725656 23157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:51.726568 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 23089
I20260926 16:38:51.726708 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:38:51.727015 11898 external_mini_cluster.cc:1428] Setting key 0e3a2f2a044ccd3e8be7b5d9637d4664
I20260926 16:38:51.733009 23157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade: Bootstrap starting.
Sep 26 16:38:51 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:38:51.735834 23157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:51.736791 23157 log.cc:824] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:51.739231 23157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade: No bootstrap required, opened a new log
I20260926 16:38:51.741750 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:51.728478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.11.158.190:38391)
0926 16:38:51.728901 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:51.728926 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:51.728963 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:38:51.729602 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:51.729618 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:51.729957 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:51.730348 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:51.731157 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:51.732161 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:51.733165 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:51.733639 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:51.733806 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:51.736420 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:51.736432 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:51.736446 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:51.736480 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:51.738835 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:51.739358 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:51.739365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:51.739371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:51.739496 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:51.739816 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:51.739820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:51.739821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:51.740219 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:51.740511 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:51.740709 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 16:38:51.741016 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":145,"threads_started":1}
I20260926 16:38:51.743191 23157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a600dd81fba480c89a365387f129ade" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38391 } }
I20260926 16:38:51.743470 23157 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:51.743537 23157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a600dd81fba480c89a365387f129ade, State: Initialized, Role: FOLLOWER
I20260926 16:38:51.744001 23157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a600dd81fba480c89a365387f129ade" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38391 } }
I20260926 16:38:51.744159 23157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:38:51.744225 23157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:38:51.744320 23157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:51.745534 23157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a600dd81fba480c89a365387f129ade" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38391 } }
I20260926 16:38:51.745894 23157 leader_election.cc:302] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a600dd81fba480c89a365387f129ade; no voters:
I20260926 16:38:51.746243 23157 leader_election.cc:288] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:38:51.746364 23162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:38:51.746644 23162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [term 1 LEADER]: Becoming Leader. State: Replica: 8a600dd81fba480c89a365387f129ade, State: Running, Role: LEADER
I20260926 16:38:51.746948 23162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a600dd81fba480c89a365387f129ade" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38391 } }
I20260926 16:38:51.747509 23157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:38:51.748018 23162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a600dd81fba480c89a365387f129ade. Latest consensus state: current_term: 1 leader_uuid: "8a600dd81fba480c89a365387f129ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a600dd81fba480c89a365387f129ade" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38391 } } }
I20260926 16:38:51.748126 23162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:51.748548 23164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a600dd81fba480c89a365387f129ade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a600dd81fba480c89a365387f129ade" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38391 } } }
I20260926 16:38:51.748643 23164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade [sys.catalog]: This master's current role is: LEADER
I20260926 16:38:51.748780 23167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:38:51.752746 23167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:38:51.753175 23157 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 16:38:51.758494 23167 catalog_manager.cc:1347] Generated new cluster ID: b0304773a95d49a4a556d8e0cf4ef5bc
I20260926 16:38:51.758566 23167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:38:51.789573 23167 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:38:51.790771 23167 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:38:51.797835 23167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade: Generated new TSK 0
I20260926 16:38:51.799125 23167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:38:52.583319 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:45713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38391
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:38:52.741385 23202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:52.741746 23202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:52.741806 23202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:52.747674 23202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:52.747813 23202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:52.747926 23202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:38:52.755306 23202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.158.148:45713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:52.756685 23202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:52.757980 23202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:52.769945 23209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:52.770071 23210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:52.769941 23212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:52.770807 23202 server_base.cc:1034] running on GCE node
I20260926 16:38:52.771283 23202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:52.772068 23202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:52.775364 23202 hybrid_clock.cc:648] HybridClock initialized: now 1790440732775101 us; error 270 us; skew 500 ppm
Sep 26 16:38:52 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440732, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:52.779362 23202 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:38:52.780580 23202 webserver.cc:460] Webserver started at http://127.11.158.129:46409/ using document root <none> and password file <none>
I20260926 16:38:52.781291 23202 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:52.781422 23202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:52.781699 23202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:38:52 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440732, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:38:52.799819 23202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "4d11c98fd9b44ffaba19331a26c58ab6"
format_stamp: "Formatted at 2026-09-26 16:38:52 on dist-test-slave-p39x"
server_key: "56ae69c028d63b0402daa387eb8c950c"
server_key_iv: "64055e14852268c8462cf11d6e63ae6d"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:52.800772 23202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "4d11c98fd9b44ffaba19331a26c58ab6"
format_stamp: "Formatted at 2026-09-26 16:38:52 on dist-test-slave-p39x"
server_key: "56ae69c028d63b0402daa387eb8c950c"
server_key_iv: "64055e14852268c8462cf11d6e63ae6d"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:52.805948 23202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 16:38:52.836778 23219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:52.838680 23202 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.001s sys 0.000s
I20260926 16:38:52.838838 23202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "4d11c98fd9b44ffaba19331a26c58ab6"
format_stamp: "Formatted at 2026-09-26 16:38:52 on dist-test-slave-p39x"
server_key: "56ae69c028d63b0402daa387eb8c950c"
server_key_iv: "64055e14852268c8462cf11d6e63ae6d"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:52.838999 23202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:52.871651 23202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:52.875370 23202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:52.875597 23202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:52.876267 23202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:52.877236 23202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:52.877305 23202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:52.877399 23202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:52.877434 23202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:52.888837 23202 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:40591
I20260926 16:38:52.888866 23334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:40591 every 8 connection(s)
I20260926 16:38:52.889956 23202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:38:52.893874 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 23202
I20260926 16:38:52.894008 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:38:52.894322 11898 external_mini_cluster.cc:1428] Setting key 7c8443ea02fc112e28f089adc1a6bf26
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Sep 26 16:38:52 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440732, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:38:52.927407 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:52.892438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38391 (local address 127.11.158.129:45445)
0926 16:38:52.892982 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:52.893047 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:52.894115 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:52.901522 (+ 7407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:52.901536 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:52.901951 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:52.902582 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:52.902599 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:52.903903 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:52.903910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:52.905542 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:52.905558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:52.906345 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:52.906351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:52.906618 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:52.907373 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:52.907398 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:52.911601 (+ 4203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:52.923405 (+ 11804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:52.923414 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:52.923436 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:52.923944 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:52.924380 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:52.924383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:52.924385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:52.924662 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:52.925123 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:52.925158 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:52.925596 (+ 438us) client_negotiation.cc:769] Sending connection context
0926 16:38:52.925885 (+ 289us) client_negotiation.cc:240] Negotiation successful
0926 16:38:52.926256 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:38:52.928984 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:52.891902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:45445 (local address 127.11.158.190:38391)
0926 16:38:52.900059 (+ 8157us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:52.900067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:52.900088 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:38:52.900950 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:52.900957 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:52.901043 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:52.902080 (+ 1037us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:52.902744 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:52.903723 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:52.905720 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:52.906136 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:52.907487 (+ 1351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:52.911887 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:52.911894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:52.911912 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:52.911953 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:52.923228 (+ 11275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:52.924105 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:52.924109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:52.924111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:52.924221 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:52.924775 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:52.924777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:52.924778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:52.924990 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:52.925224 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:52.928528 (+ 3304us) server_negotiation.cc:299] Negotiation successful
0926 16:38:52.928705 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8021,"thread_start_us":94,"threads_started":1}
I20260926 16:38:52.929816 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:45713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38391
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:38:52.932381 23335 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38391
I20260926 16:38:52.932684 23335 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:52.933343 23335 heartbeater.cc:511] Master 127.11.158.190:38391 requested a full tablet report, sending...
I20260926 16:38:52.935297 23121 ts_manager.cc:194] Registered new tserver with Master: 4d11c98fd9b44ffaba19331a26c58ab6 (127.11.158.129:40591)
I20260926 16:38:52.937117 23121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:45445
Sep 26 16:38:52 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440732, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 16:38:53.098227 23341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:53.098713 23341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:53.098860 23341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:53.105966 23341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:53.106153 23341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:53.106357 23341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:38:53.115505 23341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.158.148:45713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:53.117409 23341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:53.118948 23341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:53.132236 23348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:53.133286 23347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:53.133333 23350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:53.133556 23341 server_base.cc:1034] running on GCE node
I20260926 16:38:53.134377 23341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:53.135280 23341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:53.136471 23341 hybrid_clock.cc:648] HybridClock initialized: now 1790440733136435 us; error 46 us; skew 500 ppm
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:53.140491 23341 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:38:53.141878 23341 webserver.cc:460] Webserver started at http://127.11.158.130:34851/ using document root <none> and password file <none>
I20260926 16:38:53.142729 23341 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:53.142879 23341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:53.143205 23341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:38:53.161211 23341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "70bd3e744135403bb22243f011e8cd07"
format_stamp: "Formatted at 2026-09-26 16:38:53 on dist-test-slave-p39x"
server_key: "fe471267719af3f6cbd372f626109bd8"
server_key_iv: "12ed5c369d810bfb99e5b17b2433e6f3"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:53.166908 23341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "70bd3e744135403bb22243f011e8cd07"
format_stamp: "Formatted at 2026-09-26 16:38:53 on dist-test-slave-p39x"
server_key: "fe471267719af3f6cbd372f626109bd8"
server_key_iv: "12ed5c369d810bfb99e5b17b2433e6f3"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:53.171558 23341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260926 16:38:53.199748 23357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:53.201726 23341 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.000s
I20260926 16:38:53.201959 23341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "70bd3e744135403bb22243f011e8cd07"
format_stamp: "Formatted at 2026-09-26 16:38:53 on dist-test-slave-p39x"
server_key: "fe471267719af3f6cbd372f626109bd8"
server_key_iv: "12ed5c369d810bfb99e5b17b2433e6f3"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:53.202178 23341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:53.219624 23341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:53.223552 23341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:53.223807 23341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:53.224617 23341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:53.225555 23341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:53.225625 23341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:53.225720 23341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:53.225768 23341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:53.237005 23341 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:38623
I20260926 16:38:53.237974 23341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:38:53.239094 23470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:38623 every 8 connection(s)
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:38:53.247939 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 23341
I20260926 16:38:53.248093 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:38:53.248469 11898 external_mini_cluster.cc:1428] Setting key d46d384d5bb0d9dce1f958dc0c3ab1f2
I20260926 16:38:53.252744 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.239894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:50943 (local address 127.11.158.190:38391)
0926 16:38:53.240028 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:53.240033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:53.240975 (+ 942us) server_negotiation.cc:407] Connection header received
0926 16:38:53.241961 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:53.241965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:53.242027 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:53.242129 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:53.243209 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.243735 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.245133 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.245368 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.245499 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.248017 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:53.248022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:53.248025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:53.248058 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:53.249844 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:53.250528 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:53.250535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:53.250537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:53.250625 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:53.251052 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:53.251058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:53.251063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:53.251313 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:53.251391 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:53.252148 (+ 757us) server_negotiation.cc:299] Negotiation successful
0926 16:38:53.252310 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:38:53.254498 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.240321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38391 (local address 127.11.158.130:50943)
0926 16:38:53.240824 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:53.240875 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:53.241743 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:53.242311 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:53.242317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:53.242599 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:53.243060 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.243073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.243901 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.243904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.244942 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.244963 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.245537 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.245541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.245728 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.246386 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:53.246402 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:53.247807 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:53.249979 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:53.249986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:53.250002 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:53.250393 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:53.250746 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:53.250752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:53.250757 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:53.250938 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:53.251488 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:53.251496 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:53.251827 (+ 331us) client_negotiation.cc:769] Sending connection context
0926 16:38:53.253261 (+ 1434us) client_negotiation.cc:240] Negotiation successful
0926 16:38:53.253691 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":122,"threads_started":1}
I20260926 16:38:53.255844 23471 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38391
I20260926 16:38:53.256093 23471 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:53.256739 23471 heartbeater.cc:511] Master 127.11.158.190:38391 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:38:53.269186 23121 ts_manager.cc:194] Registered new tserver with Master: 70bd3e744135403bb22243f011e8cd07 (127.11.158.130:38623)
I20260926 16:38:53.269994 23121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:50943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:38:53.300405 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:45713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38391
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:37397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440732, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
W20260926 16:38:53.541751 23476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:38:53.542160 23476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:38:53.542243 23476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:38:53.554463 23476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:38:53.554634 23476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:38:53.554776 23476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:38:53.564039 23476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:37397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.158.148:45713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:38:53.566560 23476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:38:53.567932 23476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:38:53.580504 23481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:53.580917 23484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:38:53.581892 23482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:38:53.582414 23476 server_base.cc:1034] running on GCE node
I20260926 16:38:53.583029 23476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:38:53.583776 23476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:38:53.587050 23476 hybrid_clock.cc:648] HybridClock initialized: now 1790440733586968 us; error 79 us; skew 500 ppm
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:38:53.591758 23476 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:38:53.593001 23476 webserver.cc:460] Webserver started at http://127.11.158.131:35525/ using document root <none> and password file <none>
I20260926 16:38:53.593786 23476 fs_manager.cc:359] Metadata directory not provided
I20260926 16:38:53.593924 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:38:53.594210 23476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:38:53.625214 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "cf79ab63798d4e9fab0088cc1e54dea6"
format_stamp: "Formatted at 2026-09-26 16:38:53 on dist-test-slave-p39x"
server_key: "1c44c6eae381e695d6ef9af9b7b4bd69"
server_key_iv: "74816ba04c312f984662c66aff0667ef"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:53.626081 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "cf79ab63798d4e9fab0088cc1e54dea6"
format_stamp: "Formatted at 2026-09-26 16:38:53 on dist-test-slave-p39x"
server_key: "1c44c6eae381e695d6ef9af9b7b4bd69"
server_key_iv: "74816ba04c312f984662c66aff0667ef"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:53.637549 23476 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260926 16:38:53.664858 23491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:53.666460 23476 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260926 16:38:53.666616 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "cf79ab63798d4e9fab0088cc1e54dea6"
format_stamp: "Formatted at 2026-09-26 16:38:53 on dist-test-slave-p39x"
server_key: "1c44c6eae381e695d6ef9af9b7b4bd69"
server_key_iv: "74816ba04c312f984662c66aff0667ef"
server_key_version: "kuduclusterkey@0"
I20260926 16:38:53.666776 23476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:38:53.698408 23476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:38:53.702343 23476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:38:53.702608 23476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:38:53.703496 23476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:38:53.704638 23476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:38:53.704756 23476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:53.704855 23476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:38:53.704921 23476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:38:53.715075 23476 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:35519
I20260926 16:38:53.716286 23476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:38:53.722898 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 23476
I20260926 16:38:53.723013 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:38:53.723294 11898 external_mini_cluster.cc:1428] Setting key 366eecc0c9abccbffcc5b0d39d9e9743
I20260926 16:38:53.728730 23604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:35519 every 8 connection(s)
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440733, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:38:53.746351 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.730596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:41627 (local address 127.11.158.190:38391)
0926 16:38:53.730718 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:53.730723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:53.732820 (+ 2097us) server_negotiation.cc:407] Connection header received
0926 16:38:53.734045 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:53.734048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:53.734103 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:53.734157 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:53.735867 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.736635 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.738331 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.738690 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.738771 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.741932 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:53.741936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:53.741939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:53.741968 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:53.743780 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:53.744476 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:53.744480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:53.744482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:53.744590 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:53.745013 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:53.745016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:53.745018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:53.745262 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:53.745349 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:53.746048 (+ 699us) server_negotiation.cc:299] Negotiation successful
0926 16:38:53.746176 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:38:53.748376 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.731162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38391 (local address 127.11.158.131:41627)
0926 16:38:53.732663 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:53.732723 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:53.733793 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:53.734634 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:53.734644 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:53.735015 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:38:53.735690 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.735709 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.736786 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.736792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.738188 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.738199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.738986 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.738990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.739185 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.739870 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:38:53.739890 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:38:53.741726 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:38:53.743934 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:53.743941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:53.743960 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:53.744351 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:53.744710 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:38:53.744712 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:38:53.744715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:38:53.744901 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:38:53.745420 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:38:53.745426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:38:53.745760 (+ 334us) client_negotiation.cc:769] Sending connection context
0926 16:38:53.746847 (+ 1087us) client_negotiation.cc:240] Negotiation successful
0926 16:38:53.747206 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1263,"thread_start_us":150,"threads_started":1}
I20260926 16:38:53.749899 23605 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38391
I20260926 16:38:53.750154 23605 heartbeater.cc:464] Registering TS with master...
I20260926 16:38:53.750754 23605 heartbeater.cc:511] Master 127.11.158.190:38391 requested a full tablet report, sending...
I20260926 16:38:53.751950 23121 ts_manager.cc:194] Registered new tserver with Master: cf79ab63798d4e9fab0088cc1e54dea6 (127.11.158.131:35519)
I20260926 16:38:53.752838 23121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:41627
I20260926 16:38:53.761091 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:38:53.774989 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.763708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.11.158.190:38391)
0926 16:38:53.763830 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:53.763835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:53.764522 (+ 687us) server_negotiation.cc:407] Connection header received
0926 16:38:53.764736 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:53.764741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:53.764792 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:53.764844 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:38:53.765873 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.766740 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.767954 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.768270 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.768389 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.770955 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:38:53.770960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:38:53.770962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:38:53.770997 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:38:53.772988 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:53.773523 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:53.773527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:53.773529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:53.773630 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:38:53.773945 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:38:53.773948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:38:53.773950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:38:53.774181 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:38:53.774281 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:53.774681 (+ 400us) server_negotiation.cc:299] Negotiation successful
0926 16:38:53.774816 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:38:53.779338 23121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:38:53.784143 23121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:38:53.819783 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.796972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:40591 (local address 127.0.0.1:47086)
0926 16:38:53.802412 (+ 5440us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:53.802476 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:53.802625 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:53.806239 (+ 3614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:53.806242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:53.806256 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:53.806333 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.806338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.809959 (+ 3621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.809962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.812416 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.812431 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.819143 (+ 6712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.819147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.819299 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.819344 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 16:38:53.819467 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 16:38:53.819609 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5247,"thread_start_us":111,"threads_started":1}
I20260926 16:38:53.820673 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.799515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:35519 (local address 127.0.0.1:54828)
0926 16:38:53.802866 (+ 3351us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:53.802880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:53.802959 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:53.805957 (+ 2998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:53.805964 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:53.805986 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:53.806128 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.806134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.813200 (+ 7066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.813203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.815129 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.815145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.820182 (+ 5037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.820186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.820284 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.820310 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:38:53.820441 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 16:38:53.820526 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3289,"thread_start_us":126,"threads_started":1}
I20260926 16:38:53.822851 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.800007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.11.158.131:35519)
0926 16:38:53.803263 (+ 3256us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:53.803275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:53.803302 (+ 27us) server_negotiation.cc:407] Connection header received
0926 16:38:53.803383 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:53.803389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:53.805601 (+ 2212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:53.805753 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:53.806456 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.813042 (+ 6586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.815432 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.817481 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.817619 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.817777 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:53.822538 (+ 4761us) server_negotiation.cc:299] Negotiation successful
0926 16:38:53.822681 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3137,"thread_start_us":138,"threads_started":1}
I20260926 16:38:53.823472 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.797965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.11.158.129:40591)
0926 16:38:53.804721 (+ 6756us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:53.804729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:53.804752 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:38:53.804825 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:53.804830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:53.805080 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:53.805241 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:53.807881 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.809795 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.817861 (+ 8066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.818758 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.818901 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.819061 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:53.823176 (+ 4115us) server_negotiation.cc:299] Negotiation successful
0926 16:38:53.823316 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6642,"thread_start_us":120,"threads_started":1}
I20260926 16:38:53.825165 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.801781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38623 (local address 127.0.0.1:46182)
0926 16:38:53.804369 (+ 2588us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:53.804382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:53.804466 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:53.813295 (+ 8829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:53.813298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:53.813319 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:53.820523 (+ 7204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.820532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.822197 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.822201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.823314 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:53.823327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.824740 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.824744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:53.824847 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.824872 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:38:53.824946 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 16:38:53.825023 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2503,"mutex_wait_us":7034,"thread_start_us":92,"threads_started":1}
I20260926 16:38:53.825925 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:53.801954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46182 (local address 127.11.158.130:38623)
0926 16:38:53.812592 (+ 10638us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:53.812599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:53.812629 (+ 30us) server_negotiation.cc:407] Connection header received
0926 16:38:53.812717 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:53.812723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:53.812973 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:53.813139 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:53.820696 (+ 7557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.822048 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.823498 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:53.824384 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:53.824525 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:53.824668 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:53.825664 (+ 996us) server_negotiation.cc:299] Negotiation successful
0926 16:38:53.825782 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10493,"thread_start_us":93,"threads_started":1}
I20260926 16:38:53.827198 23539 tablet_service.cc:1467] Processing CreateTablet for tablet 26f2164057c84463bdcc34c3658ff593 (DEFAULT_TABLE table=test-table [id=d90070b011b745489d5e83a231fb408d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:38:53.829608 23539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f2164057c84463bdcc34c3658ff593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:53.829243 23387 tablet_service.cc:1467] Processing CreateTablet for tablet 26f2164057c84463bdcc34c3658ff593 (DEFAULT_TABLE table=test-table [id=d90070b011b745489d5e83a231fb408d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:38:53.830318 23387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f2164057c84463bdcc34c3658ff593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:53.839283 23624 tablet_bootstrap.cc:492] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: Bootstrap starting.
I20260926 16:38:53.841939 23624 tablet_bootstrap.cc:654] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:53.841463 23251 tablet_service.cc:1467] Processing CreateTablet for tablet 26f2164057c84463bdcc34c3658ff593 (DEFAULT_TABLE table=test-table [id=d90070b011b745489d5e83a231fb408d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:38:53.842798 23251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f2164057c84463bdcc34c3658ff593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:38:53.848608 23624 log.cc:824] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:53.855767 23626 tablet_bootstrap.cc:492] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6: Bootstrap starting.
I20260926 16:38:53.858628 23626 tablet_bootstrap.cc:654] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:53.859206 23627 tablet_bootstrap.cc:492] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Bootstrap starting.
I20260926 16:38:53.862380 23624 tablet_bootstrap.cc:492] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: No bootstrap required, opened a new log
I20260926 16:38:53.862707 23624 ts_tablet_manager.cc:1397] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.003s
I20260926 16:38:53.862674 23626 log.cc:824] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:53.865435 23627 tablet_bootstrap.cc:654] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Neither blocks nor log segments found. Creating new log.
I20260926 16:38:53.866479 23627 log.cc:824] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Log is configured to *not* fsync() on all Append() calls
I20260926 16:38:53.867301 23624 raft_consensus.cc:348] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:53.867801 23624 raft_consensus.cc:374] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:53.867919 23624 raft_consensus.cc:729] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70bd3e744135403bb22243f011e8cd07, State: Initialized, Role: FOLLOWER
I20260926 16:38:53.868577 23624 consensus_queue.cc:260] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:53.869100 23626 tablet_bootstrap.cc:492] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6: No bootstrap required, opened a new log
I20260926 16:38:53.869380 23626 ts_tablet_manager.cc:1397] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260926 16:38:53.869836 23624 ts_tablet_manager.cc:1428] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:38:53.870249 23471 heartbeater.cc:503] Master 127.11.158.190:38391 was elected leader, sending a full tablet report...
I20260926 16:38:53.874514 23626 raft_consensus.cc:348] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:53.874948 23626 raft_consensus.cc:374] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:53.875059 23626 raft_consensus.cc:729] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d11c98fd9b44ffaba19331a26c58ab6, State: Initialized, Role: FOLLOWER
I20260926 16:38:53.875691 23626 consensus_queue.cc:260] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:53.878369 23626 ts_tablet_manager.cc:1428] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260926 16:38:53.878855 23335 heartbeater.cc:503] Master 127.11.158.190:38391 was elected leader, sending a full tablet report...
I20260926 16:38:53.884192 23627 tablet_bootstrap.cc:492] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: No bootstrap required, opened a new log
I20260926 16:38:53.884516 23627 ts_tablet_manager.cc:1397] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.000s
I20260926 16:38:53.890056 23627 raft_consensus.cc:348] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:53.890465 23627 raft_consensus.cc:374] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:38:53.890558 23627 raft_consensus.cc:729] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf79ab63798d4e9fab0088cc1e54dea6, State: Initialized, Role: FOLLOWER
I20260926 16:38:53.891186 23627 consensus_queue.cc:260] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:53.892472 23627 ts_tablet_manager.cc:1428] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260926 16:38:53.892920 23605 heartbeater.cc:503] Master 127.11.158.190:38391 was elected leader, sending a full tablet report...
Sep 26 16:38:53 dist-test-slave-p39x krb5kdc[22645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440732, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
W20260926 16:38:53.904175 23336 tablet.cc:2369] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:38:53.980129 23606 tablet.cc:2369] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:38:53.989682 23472 tablet.cc:2369] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:38:54.105455 23632 raft_consensus.cc:482] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:38:54.105751 23632 raft_consensus.cc:504] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:54.107297 23632 leader_election.cc:288] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf79ab63798d4e9fab0088cc1e54dea6 (127.11.158.131:35519), 70bd3e744135403bb22243f011e8cd07 (127.11.158.130:38623)
I20260926 16:38:54.117709 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:54.107998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:56399 (local address 127.11.158.130:38623)
0926 16:38:54.108112 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:54.108117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:54.108697 (+ 580us) server_negotiation.cc:407] Connection header received
0926 16:38:54.108899 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:54.108903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:54.108975 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:54.109024 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:54.109442 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.110180 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.111773 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.112412 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.112508 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:54.112563 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:54.117454 (+ 4891us) server_negotiation.cc:299] Negotiation successful
0926 16:38:54.117563 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 16:38:54.118430 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:54.107528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38623 (local address 127.11.158.129:56399)
0926 16:38:54.108599 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:54.108615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:54.108781 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:54.109192 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:54.109195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:54.109214 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:54.109312 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:54.109318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.110332 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.110335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:54.111627 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:54.111639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.117117 (+ 5478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.117121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:54.117252 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:54.117285 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 16:38:54.118161 (+ 876us) client_negotiation.cc:240] Negotiation successful
0926 16:38:54.118259 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":106,"threads_started":1}
I20260926 16:38:54.119309 23423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f2164057c84463bdcc34c3658ff593" candidate_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70bd3e744135403bb22243f011e8cd07" is_pre_election: true
I20260926 16:38:54.119681 23423 raft_consensus.cc:2454] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d11c98fd9b44ffaba19331a26c58ab6 in term 0.
I20260926 16:38:54.120455 23223 leader_election.cc:302] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d11c98fd9b44ffaba19331a26c58ab6, 70bd3e744135403bb22243f011e8cd07; no voters:
I20260926 16:38:54.120792 23632 raft_consensus.cc:2781] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:38:54.120863 23632 raft_consensus.cc:482] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:38:54.120903 23632 raft_consensus.cc:3037] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:54.122411 23632 raft_consensus.cc:504] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:54.122790 23632 leader_election.cc:288] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [CANDIDATE]: Term 1 election: Requested vote from peers cf79ab63798d4e9fab0088cc1e54dea6 (127.11.158.131:35519), 70bd3e744135403bb22243f011e8cd07 (127.11.158.130:38623)
I20260926 16:38:54.122895 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:54.107720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:46403 (local address 127.11.158.131:35519)
0926 16:38:54.107851 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:54.107856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:54.118394 (+ 10538us) server_negotiation.cc:407] Connection header received
0926 16:38:54.118616 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:54.118620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:54.118673 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:54.118723 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:38:54.119140 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.119883 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.121491 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.122180 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.122278 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:54.122333 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:54.122685 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 16:38:54.122770 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:38:54.123325 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:54.107528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:35519 (local address 127.11.158.129:46403)
0926 16:38:54.118287 (+ 10759us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:54.118304 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:38:54.118482 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:38:54.118873 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:38:54.118876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:38:54.118895 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:38:54.119004 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:54.119011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.120012 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.120016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:54.121321 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:38:54.121334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.122430 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.122434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:38:54.122539 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:54.122563 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:38:54.123099 (+ 536us) client_negotiation.cc:240] Negotiation successful
0926 16:38:54.123179 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10667,"mutex_wait_us":374,"thread_start_us":112,"threads_started":1}
I20260926 16:38:54.123472 23423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f2164057c84463bdcc34c3658ff593" candidate_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70bd3e744135403bb22243f011e8cd07"
I20260926 16:38:54.123606 23423 raft_consensus.cc:3037] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:54.124950 23551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f2164057c84463bdcc34c3658ff593" candidate_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" is_pre_election: true
I20260926 16:38:54.125208 23423 raft_consensus.cc:2454] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d11c98fd9b44ffaba19331a26c58ab6 in term 1.
I20260926 16:38:54.125289 23551 raft_consensus.cc:2454] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d11c98fd9b44ffaba19331a26c58ab6 in term 0.
I20260926 16:38:54.125573 23223 leader_election.cc:302] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d11c98fd9b44ffaba19331a26c58ab6, 70bd3e744135403bb22243f011e8cd07; no voters:
I20260926 16:38:54.125624 23550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f2164057c84463bdcc34c3658ff593" candidate_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf79ab63798d4e9fab0088cc1e54dea6"
I20260926 16:38:54.125771 23632 raft_consensus.cc:2781] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:38:54.125736 23550 raft_consensus.cc:3037] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:38:54.126005 23632 raft_consensus.cc:686] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [term 1 LEADER]: Becoming Leader. State: Replica: 4d11c98fd9b44ffaba19331a26c58ab6, State: Running, Role: LEADER
I20260926 16:38:54.126367 23632 consensus_queue.cc:237] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } }
I20260926 16:38:54.127002 23550 raft_consensus.cc:2454] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d11c98fd9b44ffaba19331a26c58ab6 in term 1.
I20260926 16:38:54.129776 23120 catalog_manager.cc:5495] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d11c98fd9b44ffaba19331a26c58ab6 (127.11.158.129). New cstate: current_term: 1 leader_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d11c98fd9b44ffaba19331a26c58ab6" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 } health_report { overall_health: UNKNOWN } } }
I20260926 16:38:54.172408 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:54.167175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.11.158.129:40591)
0926 16:38:54.167356 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 16:38:54.167364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:38:54.167385 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:38:54.167495 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:38:54.167500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:38:54.167598 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:38:54.167737 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:38:54.168160 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.168969 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.170178 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:38:54.170497 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:38:54.170607 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:38:54.171017 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:38:54.171583 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:38:54.171713 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 16:38:54.172151 (+ 438us) server_negotiation.cc:299] Negotiation successful
0926 16:38:54.172215 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:38:54.185343 23423 raft_consensus.cc:1261] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 1 FOLLOWER]: Refusing update from remote peer 4d11c98fd9b44ffaba19331a26c58ab6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:38:54.186230 23637 consensus_queue.cc:1035] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70bd3e744135403bb22243f011e8cd07" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:38:54.186872 23550 raft_consensus.cc:1261] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 1 FOLLOWER]: Refusing update from remote peer 4d11c98fd9b44ffaba19331a26c58ab6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:38:54.187559 23637 consensus_queue.cc:1035] T 26f2164057c84463bdcc34c3658ff593 P 4d11c98fd9b44ffaba19331a26c58ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf79ab63798d4e9fab0088cc1e54dea6" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:38:54.199949 23642 mvcc.cc:205] Tried to move back new op lower bound from 7333645247213121536 to 7333645246984900608. Current Snapshot: MvccSnapshot[applied={T|T < 7333645247213121536}]
I20260926 16:38:54.202427 23645 mvcc.cc:205] Tried to move back new op lower bound from 7333645247213121536 to 7333645246984900608. Current Snapshot: MvccSnapshot[applied={T|T < 7333645247213121536}]
I20260926 16:38:54.208794 23644 mvcc.cc:205] Tried to move back new op lower bound from 7333645247213121536 to 7333645246984900608. Current Snapshot: MvccSnapshot[applied={T|T < 7333645247213121536}]
I20260926 16:38:54.215819 23121 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:38:54.216009 23121 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:38:54.218959 23121 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a600dd81fba480c89a365387f129ade: Sending DeleteTablet for 3 replicas of tablet 26f2164057c84463bdcc34c3658ff593
I20260926 16:38:54.219828 23251 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f2164057c84463bdcc34c3658ff593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:38:54 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:47086
I20260926 16:38:54.220945 23539 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f2164057c84463bdcc34c3658ff593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:38:54 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:54828
I20260926 16:38:54.221925 23387 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f2164057c84463bdcc34c3658ff593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:38:54 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:46182
I20260926 16:38:54.223318 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 23202
I20260926 16:38:54.225419 23658 tablet_replica.cc:330] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: stopping tablet replica
I20260926 16:38:54.225807 23658 raft_consensus.cc:2229] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:38:54.226074 23657 tablet_replica.cc:330] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: stopping tablet replica
I20260926 16:38:54.226467 23658 raft_consensus.cc:2258] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:38:54.226794 23657 raft_consensus.cc:2229] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:38:54.227090 23657 raft_consensus.cc:2258] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:38:54.228953 23657 ts_tablet_manager.cc:1905] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:38:54.230284 23658 ts_tablet_manager.cc:1905] T 26f2164057c84463bdcc34c3658ff593 P 70bd3e744135403bb22243f011e8cd07: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:38:54.233428 23105 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:40591 (error 108)
I20260926 16:38:54.233719 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 23341
I20260926 16:38:54.236720 23657 ts_tablet_manager.cc:1918] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:38:54.236852 23657 log.cc:1196] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/26f2164057c84463bdcc34c3658ff593
I20260926 16:38:54.237126 23657 ts_tablet_manager.cc:1939] T 26f2164057c84463bdcc34c3658ff593 P cf79ab63798d4e9fab0088cc1e54dea6: Deleting consensus metadata
I20260926 16:38:54.238380 23107 catalog_manager.cc:4841] TS cf79ab63798d4e9fab0088cc1e54dea6 (127.11.158.131:35519): tablet 26f2164057c84463bdcc34c3658ff593 (table test-table [id=d90070b011b745489d5e83a231fb408d]) successfully deleted
I20260926 16:38:54.239023 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:54.238730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:40591 (local address 127.0.0.1:47100)
0926 16:38:54.238866 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:54.238936 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:40591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 16:38:54.239231 23105 catalog_manager.cc:4584] TS 4d11c98fd9b44ffaba19331a26c58ab6 (127.11.158.129:40591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26f2164057c84463bdcc34c3658ff593: Network error: Client connection negotiation failed: client connection to 127.11.158.129:40591: connect: Connection refused (error 111)
I20260926 16:38:54.240550 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 23476
I20260926 16:38:54.242296 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:38:54.240590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38623 (local address 127.0.0.1:46194)
0926 16:38:54.242144 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0926 16:38:54.242215 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.130:38623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1461}
I20260926 16:38:54.246116 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 23089
I20260926 16:38:54.253723 11898 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 16:38:54.599345 11898 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 16:38:54.599480 11898 mini_ranger.cc:66] Stopping Ranger...
I20260926 16:38:54.945228 11898 mini_ranger.cc:68] Stopped Ranger
2026-09-26 16:38:54.945 UTC [22675] LOG: received smart shutdown request
2026-09-26 16:38:54.950 UTC [22675] LOG: background worker "logical replication launcher" (PID 22684) exited with exit code 1
2026-09-26 16:38:54.950 UTC [22679] LOG: shutting down
2026-09-26 16:38:55.147 UTC [22675] LOG: database system is shut down
2026-09-26T16:38:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46878 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:38:55 dist-test-slave-p39x krb5kdc[23668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:38:55 dist-test-slave-p39x krb5kdc[23668](info): set up 2 sockets
Sep 26 16:38:55 dist-test-slave-p39x krb5kdc[23668](info): commencing operation
krb5kdc: starting...
W20260926 16:38:56.317103 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:38:56 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:38:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:38:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/rangeradmin_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/rangeradmin_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/rangerlookup_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/rangerlookup_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
I20260926 16:38:56.490921 11898 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskupRbDL/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/postgres -l logfile start
2026-09-26 16:38:59.081 UTC [23698] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 16:38:59.081 UTC [23698] LOG: listening on IPv4 address "127.11.158.148", port 47341
2026-09-26 16:38:59.085 UTC [23698] LOG: listening on Unix socket "/tmp/.s.PGSQL.47341"
2026-09-26 16:38:59.123 UTC [23701] LOG: database system was shut down at 2026-09-26 16:38:57 UTC
2026-09-26 16:38:59.127 UTC [23698] LOG: database system is ready to accept connections
I20260926 16:39:00.069284 11898 mini_postgres.cc:96] Postgres bound to 47341
2026-09-26 16:39:00.075 UTC [23709] FATAL: database "slave" does not exist
127.11.158.148:47341 - accepting connections
I20260926 16:39:00.075958 11898 mini_ranger.cc:161] Starting Ranger...
I20260926 16:39:00.094401 11898 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 16:39:00.602324 11898 mini_ranger.cc:87] Created ranger Postgres database
I20260926 16:39:00.602411 11898 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-admin
2026-09-26 16:39:00,877 [I] DB FLAVOR :POSTGRES
2026-09-26 16:39:00,878 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:39:00,878 [I] Checking connection..
2026-09-26 16:39:00,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 16:39:01,132 [I] Checking connection passed.
2026-09-26 16:39:01,133 [I] --------- Verifying version history table ---------
2026-09-26 16:39:01,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:39:01,381 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:39:01,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:39:01,618 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:39:01,618 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 16:39:01,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 16:39:01.857 UTC [23773] WARNING: there is no transaction in progress
2026-09-26 16:39:01,864 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 16:39:01,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:39:02,129 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 16:39:02,130 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:39:02,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 16:39:02,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 16:39:02,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-p39x.c.gcp-upstream.internal','N') ;"
2026-09-26 16:39:02,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:39:03,109 [I] Table x_portal_user does not exist in database ranger
2026-09-26 16:39:03,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:39:03,372 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 16:39:03,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:39:03,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:39:03,880 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 16:39:03,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 16:39:04.887 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:04.905 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:04.923 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:04.940 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:04.961 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:04.983 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.051 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.058 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.075 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.086 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.097 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.113 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.124 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.131 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.140 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05.148 UTC [23881] WARNING: there is no transaction in progress
2026-09-26 16:39:05,867 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 16:39:05,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 16:39:06,110 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 16:39:06,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:39:06,385 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 16:39:06,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:39:06,638 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 16:39:06,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:39:06,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:39:07,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-p39x.c.gcp-upstream.internal';"
2026-09-26 16:39:07,359 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 16:39:07,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 16:39:07,622 [I] DB_PATCHES have already been applied
I20260926 16:39:07.628419 11898 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-admin:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskupRbDL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 16:39:07.628505 11898 mini_ranger.cc:193] Using host: 127.11.158.148
I20260926 16:39:07.632059 11898 mini_ranger.cc:227] Ranger admin URL: http://127.11.158.148:57449
Sep 26, 2026 4:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 4:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 4:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 4:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 4:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 16:39:08.759423 11898 mini_ranger.cc:160] Time spent starting Ranger: real 8.683s user 0.000s sys 0.010s
Sep 26, 2026 4:39:29 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 4:39:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Sep 26, 2026 4:39:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 4:39:29 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 4:39:30 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 16:39:31.130497 11898 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.158.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.158.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/rangerkms_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.158.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/rangerkms_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.158.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.158.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.158.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/HTTP_127.11.158.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 16:39:31 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440771, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 16:39:31.232860 11898 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 16:39:31.242324 11898 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 16:39:32.229871 11898 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 16:39:32.229943 11898 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms
I20260926 16:39:32.229956 11898 mini_ranger_kms.cc:227] Using postgres at 127.11.158.148:47341
2026-09-26 16:39:32,408 [I] DB FLAVOR :POSTGRES
2026-09-26 16:39:32,408 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:39:32,409 [I] Checking connection
2026-09-26 16:39:32,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 16:39:32,667 [I] connection success
2026-09-26 16:39:32,667 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 16:39:32,668 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 16:39:32,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 16:39:32,899 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 16:39:32,899 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:39:32,899 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 16:39:32,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.158.148:47341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 16:39:33,173 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 16:39:33.675607 11898 mini_ranger_kms.cc:310] Created kms service
I20260926 16:39:33.893453 11898 mini_ranger_kms.cc:326] Created kudu user
I20260926 16:39:33.945137 11898 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 16:39:33 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440771, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:34.368872 11898 mini_ranger_kms.cc:384] Added ranger policy
I20260926 16:39:34.369033 11898 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms:/tmp/dist-test-taskupRbDL/build/debug/bin/postgresql.jar:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskupRbDL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 16:39:34.369055 11898 mini_ranger_kms.cc:240] Using host: 127.11.158.148
I20260926 16:39:34.373392 11898 mini_ranger_kms.cc:276] Ranger KMS PID: 24061
I20260926 16:39:34.373508 11898 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.158.148:41865
Sep 26, 2026 4:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 4:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 4:39:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 4:39:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 16:39:36.640853 11898 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.408s user 0.017s sys 0.143s
I20260926 16:39:36.640990 11898 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 16:39:36.641096 11898 mini_ranger_kms.cc:401] 127.11.158.148:41865/kms/v1/keys
Sep 26 16:39:36 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440776, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.158.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 16:39:37 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440776, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.158.148@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
Sep 26 16:39:37 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440776, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.158.148@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
Sep 26 16:39:39 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:39:40.009131 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40427
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:40427
--ranger_config_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:40.123972 24112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:40.124231 24112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:40.124270 24112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:40.128396 24112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:40.128459 24112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:40.128479 24112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:40.128494 24112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:40.128554 24112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:40.134070 24112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.158.148:41865/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:40427
--ranger_config_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:40.135185 24112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:40.136219 24112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:40.145025 24117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:40.145099 24112 server_base.cc:1034] running on GCE node
W20260926 16:39:40.145169 24120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:40.145205 24118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:40.145946 24112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:40.147048 24112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:40.148214 24112 hybrid_clock.cc:648] HybridClock initialized: now 1790440780148189 us; error 39 us; skew 500 ppm
Sep 26 16:39:40 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440780, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:40.155767 24112 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:39:40.156875 24112 webserver.cc:460] Webserver started at http://127.11.158.190:44833/ using document root <none> and password file <none>
I20260926 16:39:40.157392 24112 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:40.157444 24112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:40.157621 24112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:39:40 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440780, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:40.215523 24112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "450ddfb0f14c4c87a8aa72932094fa2e"
format_stamp: "Formatted at 2026-09-26 16:39:40 on dist-test-slave-p39x"
server_key: "9ffac3b2b9f57f96a3e7aad40afdefbc"
server_key_iv: "43e7bf5108b15d44a14f99a79b207521"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:40.216199 24112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "450ddfb0f14c4c87a8aa72932094fa2e"
format_stamp: "Formatted at 2026-09-26 16:39:40 on dist-test-slave-p39x"
server_key: "9ffac3b2b9f57f96a3e7aad40afdefbc"
server_key_iv: "43e7bf5108b15d44a14f99a79b207521"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:40.220268 24112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:39:40.239113 24127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:40.240694 24112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:39:40.240890 24112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "450ddfb0f14c4c87a8aa72932094fa2e"
format_stamp: "Formatted at 2026-09-26 16:39:40 on dist-test-slave-p39x"
server_key: "9ffac3b2b9f57f96a3e7aad40afdefbc"
server_key_iv: "43e7bf5108b15d44a14f99a79b207521"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:40.241036 24112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:40.267264 24112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:40.270637 24112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:40.270855 24112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:40.278676 24112 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:40427
I20260926 16:39:40.278685 24179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:40427 every 8 connection(s)
I20260926 16:39:40.279661 24112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:39:40.282528 24180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:40.287377 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24112
I20260926 16:39:40.287475 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:39:40.287778 11898 external_mini_cluster.cc:1428] Setting key b5d0e99893df55bc89cd80fe20d7c596
I20260926 16:39:40.288154 24180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Bootstrap starting.
I20260926 16:39:40.290501 24180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:40.291357 24180 log.cc:824] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:40.293717 24180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: No bootstrap required, opened a new log
Sep 26 16:39:40 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:40.298146 24180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:40.298365 24180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:40.298518 24180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 450ddfb0f14c4c87a8aa72932094fa2e, State: Initialized, Role: FOLLOWER
I20260926 16:39:40.299053 24180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:40.299238 24180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:39:40.299304 24180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:39:40.299407 24180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:40.300572 24180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:40.300966 24180 leader_election.cc:302] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 450ddfb0f14c4c87a8aa72932094fa2e; no voters:
I20260926 16:39:40.301329 24180 leader_election.cc:288] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:39:40.301661 24185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:39:40.301932 24185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 LEADER]: Becoming Leader. State: Replica: 450ddfb0f14c4c87a8aa72932094fa2e, State: Running, Role: LEADER
I20260926 16:39:40.302343 24185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:40.302690 24180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:40.303647 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:40.289280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.11.158.190:40427)
0926 16:39:40.289705 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:40.289722 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:40.289757 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:39:40.290388 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:40.290406 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:40.290794 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:40.291085 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:40.292041 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:40.293201 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:40.294417 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:40.294916 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:40.295046 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:40.297933 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:40.297939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:40.297953 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:40.297980 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:40.299912 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:40.300475 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:40.300483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:40.300492 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:40.300634 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:40.300989 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:40.300994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:40.300995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:40.301536 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:40.301740 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:40.302361 (+ 621us) server_negotiation.cc:299] Negotiation successful
0926 16:39:40.302647 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":146,"threads_started":1}
I20260926 16:39:40.304798 24186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 450ddfb0f14c4c87a8aa72932094fa2e. Latest consensus state: current_term: 1 leader_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } } }
I20260926 16:39:40.304919 24186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:40.305054 24180 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 16:39:40.305320 24187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } } }
I20260926 16:39:40.305474 24187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:40.307830 24199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:39:40.311461 24199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:39:40.318826 24199 catalog_manager.cc:1347] Generated new cluster ID: cc8733eb936f49babfd1e462ef76e6e8
I20260926 16:39:40.318917 24199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:39:40.353663 24199 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:39:40.354970 24199 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:39:40.363533 24199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Generated new TSK 0
I20260926 16:39:40.364476 24199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:39:41.044610 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40427
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:41.179575 24225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:41.179935 24225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:41.180002 24225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:41.186368 24225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:41.186450 24225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:41.186543 24225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:39:41.194641 24225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.158.148:41865/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:41.196019 24225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:41.197397 24225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:41.209952 24232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:41.210053 24233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:41.209930 24235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:41.210449 24225 server_base.cc:1034] running on GCE node
I20260926 16:39:41.210860 24225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:41.211522 24225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:41.212795 24225 hybrid_clock.cc:648] HybridClock initialized: now 1790440781212755 us; error 53 us; skew 500 ppm
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:41.216450 24225 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:39:41.217769 24225 webserver.cc:460] Webserver started at http://127.11.158.129:43465/ using document root <none> and password file <none>
I20260926 16:39:41.218544 24225 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:41.218644 24225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:41.218955 24225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:41.236104 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "78aafdb9932b40e4933a0fe1f65926c1"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "4363d32a1dae6b9cc18877c8f59de732"
server_key_iv: "80fd014030f82bb5b1c02dfa0e544415"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.236831 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "78aafdb9932b40e4933a0fe1f65926c1"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "4363d32a1dae6b9cc18877c8f59de732"
server_key_iv: "80fd014030f82bb5b1c02dfa0e544415"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.241866 24225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 16:39:41.260380 24242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:41.262481 24225 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260926 16:39:41.262639 24225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "78aafdb9932b40e4933a0fe1f65926c1"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "4363d32a1dae6b9cc18877c8f59de732"
server_key_iv: "80fd014030f82bb5b1c02dfa0e544415"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.262779 24225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:41.303516 24225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:41.307490 24225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:41.307715 24225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:41.308459 24225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:41.309516 24225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:41.309592 24225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:41.309648 24225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:41.309722 24225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 16:39:41.320062 24225 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:33419
I20260926 16:39:41.321451 24225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:39:41.322000 24357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:33419 every 8 connection(s)
I20260926 16:39:41.323460 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24225
I20260926 16:39:41.323578 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:39:41.323900 11898 external_mini_cluster.cc:1428] Setting key 6949f900378441b6eba25de2dfb7cd18
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:39:41.340281 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:41.324675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40427 (local address 127.11.158.129:55257)
0926 16:39:41.325172 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:41.325226 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:41.326224 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:41.327283 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:41.327294 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:41.327669 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:41.328393 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:41.328418 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.329571 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.329576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:41.330645 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:41.330656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.331240 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.331244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:41.331424 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:41.332130 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:41.332146 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:41.334130 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:41.336761 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:41.336771 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:41.336791 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:41.337133 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:41.337467 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:41.337470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:41.337472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:41.337671 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:41.338186 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:41.338194 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:41.338539 (+ 345us) client_negotiation.cc:769] Sending connection context
0926 16:39:41.338754 (+ 215us) client_negotiation.cc:240] Negotiation successful
0926 16:39:41.339175 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":140,"threads_started":1}
I20260926 16:39:41.341238 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:41.325910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:55257 (local address 127.11.158.190:40427)
0926 16:39:41.326789 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:41.326799 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:41.326812 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:39:41.326849 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:41.326852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:41.326899 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:41.326988 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:41.328577 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.329399 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.330819 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.331083 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.331203 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:41.334328 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:41.334332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:41.334337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:41.334368 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:41.336614 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:41.337258 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:41.337262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:41.337264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:41.337358 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:41.337793 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:41.337796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:41.337798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:41.338032 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:41.338130 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:41.340909 (+ 2779us) server_negotiation.cc:299] Negotiation successful
0926 16:39:41.341067 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":111,"threads_started":1}
I20260926 16:39:41.343473 24358 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40427
I20260926 16:39:41.343748 24358 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:41.344406 24358 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:41.346235 24144 ts_manager.cc:194] Registered new tserver with Master: 78aafdb9932b40e4933a0fe1f65926c1 (127.11.158.129:33419)
I20260926 16:39:41.347797 24144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:55257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:39:41.367861 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40427
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 16:39:41.513733 24364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:41.514168 24364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:41.514279 24364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:41.520499 24364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:41.520581 24364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:41.520668 24364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:39:41.528620 24364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.158.148:41865/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:41.530135 24364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:41.531505 24364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:41.543938 24373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:41.544291 24364 server_base.cc:1034] running on GCE node
W20260926 16:39:41.544375 24371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:41.544672 24370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:41.545214 24364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:41.546057 24364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:41.549379 24364 hybrid_clock.cc:648] HybridClock initialized: now 1790440781549340 us; error 49 us; skew 500 ppm
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:41.552296 24364 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:39:41.553375 24364 webserver.cc:460] Webserver started at http://127.11.158.130:36459/ using document root <none> and password file <none>
I20260926 16:39:41.554136 24364 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:41.554289 24364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:41.554592 24364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:41.572259 24364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "c4b8f3c9518d4c898efbb38b3358d402"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "e19176bad7424cb53dcaffb3a1cb50dc"
server_key_iv: "49e7bac1a820f2bb0bd03bb059210db5"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.573236 24364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "c4b8f3c9518d4c898efbb38b3358d402"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "e19176bad7424cb53dcaffb3a1cb50dc"
server_key_iv: "49e7bac1a820f2bb0bd03bb059210db5"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.578824 24364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 16:39:41.596097 24380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:41.597806 24364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:39:41.598008 24364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "c4b8f3c9518d4c898efbb38b3358d402"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "e19176bad7424cb53dcaffb3a1cb50dc"
server_key_iv: "49e7bac1a820f2bb0bd03bb059210db5"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.598160 24364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:41.617066 24364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:41.620843 24364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:41.621117 24364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:41.621882 24364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:41.622987 24364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:41.623073 24364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:41.623168 24364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:41.623217 24364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:41.635556 24364 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:37779
I20260926 16:39:41.636811 24364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:39:41.637053 24493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:37779 every 8 connection(s)
I20260926 16:39:41.637907 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24364
I20260926 16:39:41.638024 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:39:41.638324 11898 external_mini_cluster.cc:1428] Setting key cbbb5c90fd68669f17e0d5998be17af6
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:41.657378 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:41.642428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:43415 (local address 127.11.158.190:40427)
0926 16:39:41.642573 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:41.642578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:41.643725 (+ 1147us) server_negotiation.cc:407] Connection header received
0926 16:39:41.645124 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:41.645128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:41.645191 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:41.645256 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:41.647010 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.647900 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.649694 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.649938 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.650052 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:41.652987 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:41.652990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:41.652993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:41.653027 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:41.654829 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:41.655485 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:41.655489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:41.655490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:41.655588 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:41.656016 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:41.656020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:41.656022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:41.656245 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:41.656362 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:41.657062 (+ 700us) server_negotiation.cc:299] Negotiation successful
0926 16:39:41.657204 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:39:41.658483 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:41.642983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40427 (local address 127.11.158.130:43415)
0926 16:39:41.643572 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:41.643629 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:41.644870 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:41.645710 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:41.645723 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:41.646110 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:41.646834 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:41.646855 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.648063 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.648066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:41.649567 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:41.649576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:41.650088 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:41.650092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:41.650279 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:41.650893 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:41.650911 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:41.652818 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:41.654953 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:41.654962 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:41.654979 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:41.655334 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:41.655695 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:41.655701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:41.655706 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:41.655895 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:41.656412 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:41.656421 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:41.656768 (+ 347us) client_negotiation.cc:769] Sending connection context
0926 16:39:41.656975 (+ 207us) client_negotiation.cc:240] Negotiation successful
0926 16:39:41.657344 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":156,"threads_started":1}
I20260926 16:39:41.659814 24494 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40427
I20260926 16:39:41.660087 24494 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:41.660692 24494 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:41.661895 24144 ts_manager.cc:194] Registered new tserver with Master: c4b8f3c9518d4c898efbb38b3358d402 (127.11.158.130:37779)
I20260926 16:39:41.662446 24144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:43415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:39:41.669391 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40427
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
W20260926 16:39:41.832969 24499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:41.833410 24499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:41.833524 24499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:41.839685 24499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:41.839843 24499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:41.839985 24499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:39:41.848035 24499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.158.148:41865/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:41.849428 24499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:41.850701 24499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:41.863188 24504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:41.863396 24505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:41.863432 24507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:41.863902 24499 server_base.cc:1034] running on GCE node
I20260926 16:39:41.864409 24499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:41.865170 24499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:41.866413 24499 hybrid_clock.cc:648] HybridClock initialized: now 1790440781866343 us; error 85 us; skew 500 ppm
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:41.870319 24499 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:39:41.871356 24499 webserver.cc:460] Webserver started at http://127.11.158.131:44227/ using document root <none> and password file <none>
I20260926 16:39:41.872030 24499 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:41.872128 24499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:41.872376 24499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:39:41 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:41.905395 24499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "62315196b3a54c83b90fb149df18181b"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "6bfc271fc3fc405ccdd56545f06dc6a9"
server_key_iv: "e92707b529a7013c76c1ee33b69f6388"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.906265 24499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "62315196b3a54c83b90fb149df18181b"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "6bfc271fc3fc405ccdd56545f06dc6a9"
server_key_iv: "e92707b529a7013c76c1ee33b69f6388"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.912747 24499 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260926 16:39:41.948830 24514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:41.950460 24499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:41.950575 24499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "62315196b3a54c83b90fb149df18181b"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "6bfc271fc3fc405ccdd56545f06dc6a9"
server_key_iv: "e92707b529a7013c76c1ee33b69f6388"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:41.950698 24499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:41.981868 24499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:41.985538 24499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:41.985774 24499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:41.986436 24499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:41.987444 24499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:41.987546 24499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:41.987636 24499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:41.987696 24499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.000556 24499 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:38405
I20260926 16:39:42.001703 24499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:39:42.002300 24627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:38405 every 8 connection(s)
I20260926 16:39:42.008492 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24499
I20260926 16:39:42.008621 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:39:42.008924 11898 external_mini_cluster.cc:1428] Setting key 41d60d35e9d66a76e7ff4f6fda47ec83
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440781, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:42.021332 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:42.006599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:36563 (local address 127.11.158.190:40427)
0926 16:39:42.006726 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:42.006730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:42.007253 (+ 523us) server_negotiation.cc:407] Connection header received
0926 16:39:42.008814 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:42.008818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:42.008871 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:42.008923 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:42.011342 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.012225 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.013900 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.014254 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.014322 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:42.017280 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:42.017284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:42.017286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:42.017313 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:42.019041 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:42.019623 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:42.019627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:42.019629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:42.019709 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:42.020083 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:42.020086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:42.020088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:42.020292 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:42.020391 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:42.021040 (+ 649us) server_negotiation.cc:299] Negotiation successful
0926 16:39:42.021169 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:39:42.024470 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:42.005276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40427 (local address 127.11.158.131:36563)
0926 16:39:42.007091 (+ 1815us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:42.007148 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:42.008214 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:42.009592 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:42.009600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:42.010302 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:42.011183 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:42.011203 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.012429 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.012433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:42.013768 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:42.013778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.014510 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.014513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:42.014698 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:42.015363 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:42.015382 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:42.017099 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:42.019176 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:42.019181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:42.019196 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:42.019513 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:42.019807 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:42.019810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:42.019812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:42.019981 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:42.020455 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:42.020461 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:42.020750 (+ 289us) client_negotiation.cc:769] Sending connection context
0926 16:39:42.022141 (+ 1391us) client_negotiation.cc:240] Negotiation successful
0926 16:39:42.023365 (+ 1224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1587,"thread_start_us":147,"threads_started":1}
I20260926 16:39:42.025898 24628 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40427
I20260926 16:39:42.026136 24628 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:42.026669 24628 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:42.027746 24144 ts_manager.cc:194] Registered new tserver with Master: 62315196b3a54c83b90fb149df18181b (127.11.158.131:38405)
I20260926 16:39:42.028496 24144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:36563
I20260926 16:39:42.034647 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:39:42.035473 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24225
I20260926 16:39:42.044406 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24364
I20260926 16:39:42.052136 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24499
I20260926 16:39:42.059860 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24112
I20260926 16:39:42.066725 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40427
--webserver_interface=127.11.158.190
--webserver_port=44833
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:40427
--ranger_config_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:42.180533 24631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:42.180807 24631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:42.180847 24631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:42.185014 24631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:42.185081 24631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:42.185099 24631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:42.185117 24631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:42.185132 24631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:42.190613 24631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:40427
--ranger_config_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=44833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:42.191716 24631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:42.192858 24631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:42.201855 24637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:42.201949 24636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:42.201925 24639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:42.201995 24631 server_base.cc:1034] running on GCE node
I20260926 16:39:42.203087 24631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:42.204326 24631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:42.205574 24631 hybrid_clock.cc:648] HybridClock initialized: now 1790440782205554 us; error 33 us; skew 500 ppm
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:42.208413 24631 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:39:42.209389 24631 webserver.cc:460] Webserver started at http://127.11.158.190:44833/ using document root <none> and password file <none>
I20260926 16:39:42.209931 24631 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:42.209997 24631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:42.276185 24631 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:42.278425 24647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.279734 24631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:39:42.279889 24631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "450ddfb0f14c4c87a8aa72932094fa2e"
format_stamp: "Formatted at 2026-09-26 16:39:40 on dist-test-slave-p39x"
server_key: "9ffac3b2b9f57f96a3e7aad40afdefbc"
server_key_iv: "43e7bf5108b15d44a14f99a79b207521"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:42.280508 24631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:42.292062 24631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:42.294965 24631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:42.295192 24631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:42.302296 24631 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:40427
I20260926 16:39:42.302300 24699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:40427 every 8 connection(s)
I20260926 16:39:42.303362 24631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:39:42.304287 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24631
I20260926 16:39:42.304756 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:33419
--local_ip_for_outbound_sockets=127.11.158.129
--tserver_master_addrs=127.11.158.190:40427
--webserver_port=43465
--webserver_interface=127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:39:42.308387 24700 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:39:42.310050 24700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Bootstrap starting.
I20260926 16:39:42.326690 24700 log.cc:824] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:42.333674 24700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:39:42.334142 24700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Bootstrap complete.
I20260926 16:39:42.338411 24700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:42.338950 24700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 450ddfb0f14c4c87a8aa72932094fa2e, State: Initialized, Role: FOLLOWER
I20260926 16:39:42.339509 24700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:42.339661 24700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:39:42.339756 24700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:39:42.339875 24700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:39:42.341938 24700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:42.342315 24700 leader_election.cc:302] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 450ddfb0f14c4c87a8aa72932094fa2e; no voters:
I20260926 16:39:42.342682 24700 leader_election.cc:288] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:39:42.342777 24704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:39:42.343034 24704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [term 2 LEADER]: Becoming Leader. State: Replica: 450ddfb0f14c4c87a8aa72932094fa2e, State: Running, Role: LEADER
I20260926 16:39:42.343401 24704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } }
I20260926 16:39:42.344002 24700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:42.345010 24706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 450ddfb0f14c4c87a8aa72932094fa2e. Latest consensus state: current_term: 2 leader_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } } }
I20260926 16:39:42.345141 24706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:42.345471 24705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ddfb0f14c4c87a8aa72932094fa2e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40427 } } }
I20260926 16:39:42.345534 24713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:39:42.345556 24705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:42.348960 24713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:39:42.349524 24700 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 16:39:42.349758 24713 catalog_manager.cc:1259] Loaded cluster ID: cc8733eb936f49babfd1e462ef76e6e8
I20260926 16:39:42.349798 24713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:39:42.351642 24713 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:39:42.352267 24713 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Loaded TSK: 0
I20260926 16:39:42.353300 24713 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:39:42.453106 24702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:42.453485 24702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:42.453549 24702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:42.459693 24702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:42.459779 24702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:42.459872 24702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:39:42.467880 24702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:33419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=43465
--tserver_master_addrs=127.11.158.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:42.469422 24702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:42.470736 24702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:39:42.483901 24702 server_base.cc:1034] running on GCE node
W20260926 16:39:42.484189 24740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:42.484395 24737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:42.483868 24738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:42.485146 24702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:42.485881 24702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:42.487059 24702 hybrid_clock.cc:648] HybridClock initialized: now 1790440782487027 us; error 49 us; skew 500 ppm
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:42.490533 24702 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:39:42.491681 24702 webserver.cc:460] Webserver started at http://127.11.158.129:43465/ using document root <none> and password file <none>
I20260926 16:39:42.492415 24702 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:42.492517 24702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:42.515054 24702 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:39:42.524799 24748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.526571 24702 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260926 16:39:42.526708 24702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "78aafdb9932b40e4933a0fe1f65926c1"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "4363d32a1dae6b9cc18877c8f59de732"
server_key_iv: "80fd014030f82bb5b1c02dfa0e544415"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:42.527442 24702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:42.554118 24702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:42.557677 24702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:42.557919 24702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:42.558575 24702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:42.559651 24702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:42.559763 24702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.559896 24702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:42.559980 24702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.571329 24702 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:33419
I20260926 16:39:42.571344 24861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:33419 every 8 connection(s)
I20260926 16:39:42.572643 24702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:39:42.578435 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24702
I20260926 16:39:42.578912 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:37779
--local_ip_for_outbound_sockets=127.11.158.130
--tserver_master_addrs=127.11.158.190:40427
--webserver_port=36459
--webserver_interface=127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:42.599550 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:42.575788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40427 (local address 127.11.158.129:37169)
0926 16:39:42.576925 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:42.576977 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:42.577967 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:42.584262 (+ 6295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:42.584273 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:42.584685 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:42.585378 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:42.585399 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.587683 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.587687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:42.589072 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:42.589085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.590152 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.590156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:42.590353 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:42.591105 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:42.591126 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:42.592913 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:42.596062 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:42.596068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:42.596083 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:42.596438 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:42.596866 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:42.596869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:42.596872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:42.597037 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:42.597745 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:42.597752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:42.598032 (+ 280us) client_negotiation.cc:769] Sending connection context
0926 16:39:42.598205 (+ 173us) client_negotiation.cc:240] Negotiation successful
0926 16:39:42.598519 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":135,"threads_started":1}
I20260926 16:39:42.601871 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:42.575707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:37169 (local address 127.11.158.190:40427)
0926 16:39:42.582913 (+ 7206us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:42.582932 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:42.582971 (+ 39us) server_negotiation.cc:407] Connection header received
0926 16:39:42.583234 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:42.583250 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:42.583721 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:42.585649 (+ 1928us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:42.585960 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.587507 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.589236 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.589966 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.590754 (+ 788us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:42.593130 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:42.593137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:42.593153 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:42.593183 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:42.595926 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:42.596590 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:42.596596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:42.596601 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:42.596744 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:42.597154 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:42.597157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:42.597159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:42.597612 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:42.600581 (+ 2969us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:42.600733 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 16:39:42.601017 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7039,"thread_start_us":114,"threads_started":1}
I20260926 16:39:42.604789 24862 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40427
I20260926 16:39:42.605024 24862 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:42.605594 24862 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
W20260926 16:39:42.607854 24862 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:40427 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 16:39:42.607975 24862 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:42.608157 24862 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:42.609043 24862 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:42.609218 24862 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
W20260926 16:39:42.609972 24862 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:39:42.742058 24866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:42.742444 24866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:42.742524 24866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:42.748893 24866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:42.749035 24866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:42.749166 24866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:39:42.757408 24866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:37779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=36459
--tserver_master_addrs=127.11.158.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:42.758545 24866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:42.759594 24866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:42.770164 24874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:42.770187 24871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:42.770394 24872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:42.771167 24866 server_base.cc:1034] running on GCE node
I20260926 16:39:42.771730 24866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:42.772701 24866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:42.773965 24866 hybrid_clock.cc:648] HybridClock initialized: now 1790440782773934 us; error 59 us; skew 500 ppm
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:42.777015 24866 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:39:42.778229 24866 webserver.cc:460] Webserver started at http://127.11.158.130:36459/ using document root <none> and password file <none>
I20260926 16:39:42.778992 24866 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:42.779117 24866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:42.807055 24866 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:39:42.810178 24882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.811784 24866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:39:42.811900 24866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "c4b8f3c9518d4c898efbb38b3358d402"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "e19176bad7424cb53dcaffb3a1cb50dc"
server_key_iv: "49e7bac1a820f2bb0bd03bb059210db5"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:42.812631 24866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:42.853315 24866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:42.856678 24866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:42.856890 24866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:42.857551 24866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:42.858537 24866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:42.858616 24866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.858690 24866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:42.858740 24866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:42.868866 24866 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:37779
I20260926 16:39:42.870256 24866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:39:42.877946 24996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:37779 every 8 connection(s)
I20260926 16:39:42.878432 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24866
I20260926 16:39:42.878896 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.158.148:41865/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:38405
--local_ip_for_outbound_sockets=127.11.158.131
--tserver_master_addrs=127.11.158.190:40427
--webserver_port=44227
--webserver_interface=127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:39:42 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440782, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:42.897359 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:42.879762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:55699 (local address 127.11.158.190:40427)
0926 16:39:42.879951 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:42.879958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:42.881858 (+ 1900us) server_negotiation.cc:407] Connection header received
0926 16:39:42.882822 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:42.882829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:42.882914 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:42.883039 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:42.885655 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.886499 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.888605 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.889070 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.889175 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:42.892709 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:42.892713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:42.892718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:42.892750 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:42.894675 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:42.895376 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:42.895380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:42.895382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:42.895478 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:42.895938 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:42.895941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:42.895943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:42.896185 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:42.896272 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:42.897041 (+ 769us) server_negotiation.cc:299] Negotiation successful
0926 16:39:42.897185 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 16:39:42.899394 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:42.880293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40427 (local address 127.11.158.130:55699)
0926 16:39:42.881700 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:42.881740 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:42.882595 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:42.883326 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:42.883339 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:42.883749 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:42.884554 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:42.884576 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.886707 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.886714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:42.888417 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:42.888433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:42.889441 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:42.889446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:42.889722 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:42.890478 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:42.890508 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:42.892458 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:42.894834 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:42.894840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:42.894860 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:42.895244 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:42.895599 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:42.895601 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:42.895603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:42.895797 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:42.896382 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:42.896389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:42.896730 (+ 341us) client_negotiation.cc:769] Sending connection context
0926 16:39:42.897878 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0926 16:39:42.898258 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1176,"thread_start_us":199,"threads_started":1}
I20260926 16:39:42.900826 24997 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40427
I20260926 16:39:42.901098 24997 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:42.901710 24997 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
W20260926 16:39:42.903539 24997 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:40427 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 16:39:42.903671 24997 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:42.903873 24997 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:42.904814 24997 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:42.905007 24997 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
W20260926 16:39:42.905921 24997 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:39:43.021968 24999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:43.022257 24999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:43.022302 24999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:43.027359 24999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:43.027463 24999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:43.027572 24999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:39:43.034971 24999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.158.148:41865/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:38405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=44227
--tserver_master_addrs=127.11.158.190:40427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:43.036569 24999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:43.037626 24999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:43.049225 25005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:43.049147 25008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:43.049147 25006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:43.049603 24999 server_base.cc:1034] running on GCE node
I20260926 16:39:43.050103 24999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:43.050884 24999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:43.052057 24999 hybrid_clock.cc:648] HybridClock initialized: now 1790440783052023 us; error 46 us; skew 500 ppm
Sep 26 16:39:43 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440783, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:43.055039 24999 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:39:43.056119 24999 webserver.cc:460] Webserver started at http://127.11.158.131:44227/ using document root <none> and password file <none>
I20260926 16:39:43.056906 24999 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:43.057015 24999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:39:43 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440783, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:43.080775 24999 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.003s sys 0.000s
I20260926 16:39:43.084126 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:43.085695 24999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:39:43.085856 24999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "62315196b3a54c83b90fb149df18181b"
format_stamp: "Formatted at 2026-09-26 16:39:41 on dist-test-slave-p39x"
server_key: "6bfc271fc3fc405ccdd56545f06dc6a9"
server_key_iv: "e92707b529a7013c76c1ee33b69f6388"
server_key_version: "kuduclusterkey@0"
I20260926 16:39:43.086472 24999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:43.106932 24999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:43.111701 24999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:43.111970 24999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:43.112807 24999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:43.113897 24999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:43.113988 24999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:43.114068 24999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:43.114122 24999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:43.125166 24999 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:38405
I20260926 16:39:43.125195 25129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:38405 every 8 connection(s)
I20260926 16:39:43.126324 24999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:39:43.130937 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 24999
Sep 26 16:39:43 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440783, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:43.144683 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.131649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.11.158.190:40427)
0926 16:39:43.132555 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:43.132566 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:43.132580 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:39:43.132621 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:43.132624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:43.132667 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:43.133761 (+ 1094us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:43.133843 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.134584 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.135792 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.136138 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.136244 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.139342 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:43.139346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:43.139349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:43.139377 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:43.141799 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:43.142365 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:43.142368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:43.142370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:43.142468 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:43.142833 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:43.142836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:43.142838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:43.143094 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:43.143318 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:43.144244 (+ 926us) server_negotiation.cc:299] Negotiation successful
0926 16:39:43.144443 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":92,"threads_started":1}
I20260926 16:39:43.145448 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.128457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:44451 (local address 127.11.158.190:40427)
0926 16:39:43.128628 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:43.128633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:43.129605 (+ 972us) server_negotiation.cc:407] Connection header received
0926 16:39:43.130863 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:43.130866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:43.130928 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:43.131033 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:43.133135 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.133944 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.136509 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.136821 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.136926 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.140705 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:43.140713 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:43.140716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:43.140770 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:43.142766 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:43.143482 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:43.143485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:43.143487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:43.143568 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:43.144031 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:43.144034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:43.144036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:43.144273 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:43.144424 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:43.145169 (+ 745us) server_negotiation.cc:299] Negotiation successful
0926 16:39:43.145286 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:39:43.146971 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.128901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:40427 (local address 127.11.158.131:44451)
0926 16:39:43.129404 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:43.129461 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:43.130587 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:43.131250 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:43.131261 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:43.132160 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:43.132923 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.132948 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.134102 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.134105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.136378 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.136389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.137102 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.137105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.137262 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.137888 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:43.137911 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:43.140072 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:43.142912 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:43.142920 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:43.142942 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:43.143359 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:43.143677 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:43.143681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:43.143682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:43.143890 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:43.144481 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:43.144487 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:43.144880 (+ 393us) client_negotiation.cc:769] Sending connection context
0926 16:39:43.145609 (+ 729us) client_negotiation.cc:240] Negotiation successful
0926 16:39:43.146032 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":111,"threads_started":1}
I20260926 16:39:43.148161 25130 heartbeater.cc:347] Connected to a master server at 127.11.158.190:40427
I20260926 16:39:43.148439 25130 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:43.148967 25130 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
W20260926 16:39:43.150570 25130 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:40427 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 16:39:43.150697 25130 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:43.150899 25130 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:43.151932 24664 ts_manager.cc:194] Registered new tserver with Master: 62315196b3a54c83b90fb149df18181b (127.11.158.131:38405)
I20260926 16:39:43.153237 24664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:44451
Sep 26 16:39:43 dist-test-slave-p39x krb5kdc[23668](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440783, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:43.610194 24862 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:43.610520 24862 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:43.611359 24664 ts_manager.cc:194] Registered new tserver with Master: 78aafdb9932b40e4933a0fe1f65926c1 (127.11.158.129:33419)
I20260926 16:39:43.611975 24664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:37169
Sep 26 16:39:43 dist-test-slave-p39x krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440783, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for HTTP/127.11.158.148@KRBTEST.COM
I20260926 16:39:43.906141 24997 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:43.906447 24997 heartbeater.cc:511] Master 127.11.158.190:40427 requested a full tablet report, sending...
I20260926 16:39:43.907294 24663 ts_manager.cc:194] Registered new tserver with Master: c4b8f3c9518d4c898efbb38b3358d402 (127.11.158.130:37779)
I20260926 16:39:43.908058 24663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:55699
I20260926 16:39:43.922890 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:39:43.947270 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.926885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55538 (local address 127.11.158.190:40427)
0926 16:39:43.927174 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:43.927179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:43.936988 (+ 9809us) server_negotiation.cc:407] Connection header received
0926 16:39:43.937054 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:43.937057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:43.937118 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:43.937204 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:43.938272 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.939165 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.940404 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.940776 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.940881 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.942767 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:43.942771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:43.942774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:43.942805 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:43.944844 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:43.945407 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:43.945411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:43.945412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:43.945506 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:43.945881 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:43.945885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:43.945887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:43.946122 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:43.946256 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:43.946955 (+ 699us) server_negotiation.cc:299] Negotiation successful
0926 16:39:43.947083 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":107,"threads_started":1}
I20260926 16:39:43.952004 24663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:39:43.957286 24663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:39:43.977314 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.968948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:33419 (local address 127.0.0.1:41088)
0926 16:39:43.970763 (+ 1815us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:43.970812 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:43.970956 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:43.971499 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:43.971505 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:43.971529 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:43.971671 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.972004 (+ 333us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.973523 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.973526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.975352 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.975364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.976744 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.976748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.976857 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.976896 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 16:39:43.977019 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 16:39:43.977152 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1651,"thread_start_us":107,"threads_started":1}
I20260926 16:39:43.978608 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.969782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.11.158.129:33419)
0926 16:39:43.970124 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:43.970132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:43.970925 (+ 793us) server_negotiation.cc:407] Connection header received
0926 16:39:43.971113 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:43.971119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:43.971332 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:43.971752 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:43.972145 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.973391 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.975486 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.976387 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.976514 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.976682 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:43.978355 (+ 1673us) server_negotiation.cc:299] Negotiation successful
0926 16:39:43.978467 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":119,"threads_started":1}
I20260926 16:39:43.981272 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.969038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:38405 (local address 127.0.0.1:40506)
0926 16:39:43.972563 (+ 3525us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:43.972586 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:43.972673 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:43.977068 (+ 4395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:43.977071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:43.977102 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:43.977193 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.977198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.978661 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.978664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.979778 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.979789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.980909 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.980912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.980994 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.981019 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:39:43.981064 (+ 45us) client_negotiation.cc:240] Negotiation successful
0926 16:39:43.981138 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3459,"mutex_wait_us":34,"thread_start_us":112,"threads_started":1}
I20260926 16:39:43.981986 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.969200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.11.158.131:38405)
0926 16:39:43.976563 (+ 7363us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:43.976572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:43.976600 (+ 28us) server_negotiation.cc:407] Connection header received
0926 16:39:43.976681 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:43.976686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:43.976904 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:43.977352 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:43.977490 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.978524 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.979919 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.980776 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.981584 (+ 808us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.981715 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:43.981764 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 16:39:43.981854 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7245,"thread_start_us":78,"threads_started":1}
I20260926 16:39:43.982103 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.970239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:37779 (local address 127.0.0.1:39408)
0926 16:39:43.976496 (+ 6257us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:43.976511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:43.976604 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:43.977306 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:43.977308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:43.977328 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:43.977420 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.977429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.979407 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.979410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.980532 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:43.980543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.981733 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.981736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:43.981830 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.981853 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:39:43.981922 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 16:39:43.981984 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6181,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260926 16:39:43.981797 24796 tablet_service.cc:1467] Processing CreateTablet for tablet ec65c53004bb4656b04f42997be587c0 (DEFAULT_TABLE table=test-table [id=ab26a0b740a24a2fb26e311e8d97f816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:43.982565 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:43.970400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.11.158.130:37779)
0926 16:39:43.972920 (+ 2520us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:43.972927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:43.976707 (+ 3780us) server_negotiation.cc:407] Connection header received
0926 16:39:43.976792 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:43.976797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:43.977008 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:43.977150 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:43.977567 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.978595 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.980672 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:43.981420 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:43.981548 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:43.981671 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:43.982321 (+ 650us) server_negotiation.cc:299] Negotiation successful
0926 16:39:43.982423 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2402,"thread_start_us":76,"threads_started":1}
I20260926 16:39:43.983016 24796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec65c53004bb4656b04f42997be587c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:43.985133 25057 tablet_service.cc:1467] Processing CreateTablet for tablet ec65c53004bb4656b04f42997be587c0 (DEFAULT_TABLE table=test-table [id=ab26a0b740a24a2fb26e311e8d97f816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:43.985224 24931 tablet_service.cc:1467] Processing CreateTablet for tablet ec65c53004bb4656b04f42997be587c0 (DEFAULT_TABLE table=test-table [id=ab26a0b740a24a2fb26e311e8d97f816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:43.986183 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec65c53004bb4656b04f42997be587c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:43.986187 25057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec65c53004bb4656b04f42997be587c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:43.994449 25164 tablet_bootstrap.cc:492] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Bootstrap starting.
I20260926 16:39:43.996294 25165 tablet_bootstrap.cc:492] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Bootstrap starting.
I20260926 16:39:43.997596 25164 tablet_bootstrap.cc:654] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:43.998596 25165 tablet_bootstrap.cc:654] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:43.998591 25164 log.cc:824] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:43.999490 25165 log.cc:824] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:44.000931 25164 tablet_bootstrap.cc:492] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: No bootstrap required, opened a new log
I20260926 16:39:44.001145 25164 ts_tablet_manager.cc:1397] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:39:44.004959 25163 tablet_bootstrap.cc:492] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1: Bootstrap starting.
I20260926 16:39:44.006449 25164 raft_consensus.cc:348] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.006875 25164 raft_consensus.cc:374] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:44.006966 25164 raft_consensus.cc:729] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62315196b3a54c83b90fb149df18181b, State: Initialized, Role: FOLLOWER
I20260926 16:39:44.007628 25163 tablet_bootstrap.cc:654] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:44.007570 25164 consensus_queue.cc:260] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.008858 25163 log.cc:824] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:44.008952 25164 ts_tablet_manager.cc:1428] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260926 16:39:44.009392 25130 heartbeater.cc:503] Master 127.11.158.190:40427 was elected leader, sending a full tablet report...
I20260926 16:39:44.009711 25169 raft_consensus.cc:482] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:39:44.012115 25165 tablet_bootstrap.cc:492] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: No bootstrap required, opened a new log
I20260926 16:39:44.012501 25165 ts_tablet_manager.cc:1397] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260926 16:39:44.009879 25169 raft_consensus.cc:504] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.016899 25169 leader_election.cc:288] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78aafdb9932b40e4933a0fe1f65926c1 (127.11.158.129:33419), c4b8f3c9518d4c898efbb38b3358d402 (127.11.158.130:37779)
I20260926 16:39:44.019726 25163 tablet_bootstrap.cc:492] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1: No bootstrap required, opened a new log
I20260926 16:39:44.020067 25163 ts_tablet_manager.cc:1397] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260926 16:39:44.022823 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:44.017842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:35499 (local address 127.11.158.129:33419)
0926 16:39:44.017976 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:44.017981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:44.018206 (+ 225us) server_negotiation.cc:407] Connection header received
0926 16:39:44.018428 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:44.018431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:44.018478 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:44.018527 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:44.019001 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.019790 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.021392 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.022021 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.022119 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:44.022172 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:44.022600 (+ 428us) server_negotiation.cc:299] Negotiation successful
0926 16:39:44.022687 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:39:44.023566 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:44.016953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:33419 (local address 127.11.158.131:35499)
0926 16:39:44.018119 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:44.018135 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:44.018304 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:44.018683 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:44.018685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:44.018701 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:44.018885 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:44.018892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.019941 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.019944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:44.021243 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:44.021256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.022272 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.022275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:44.022377 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:44.022400 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:39:44.023314 (+ 914us) client_negotiation.cc:240] Negotiation successful
0926 16:39:44.023407 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":110,"threads_started":1}
I20260926 16:39:44.024471 24804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec65c53004bb4656b04f42997be587c0" candidate_uuid: "62315196b3a54c83b90fb149df18181b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78aafdb9932b40e4933a0fe1f65926c1" is_pre_election: true
W20260926 16:39:44.025522 25020 leader_election.cc:341] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78aafdb9932b40e4933a0fe1f65926c1 (127.11.158.129:33419): Illegal state: must be running to vote when last-logged opid is not known
I20260926 16:39:44.025571 25163 raft_consensus.cc:348] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.025926 25163 raft_consensus.cc:374] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:44.026017 25163 raft_consensus.cc:729] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78aafdb9932b40e4933a0fe1f65926c1, State: Initialized, Role: FOLLOWER
I20260926 16:39:44.026598 25163 consensus_queue.cc:260] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.027547 25165 raft_consensus.cc:348] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.027774 25163 ts_tablet_manager.cc:1428] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1: Time spent starting tablet: real 0.008s user 0.001s sys 0.004s
I20260926 16:39:44.027889 25165 raft_consensus.cc:374] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:44.028003 25165 raft_consensus.cc:729] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b8f3c9518d4c898efbb38b3358d402, State: Initialized, Role: FOLLOWER
I20260926 16:39:44.028147 24862 heartbeater.cc:503] Master 127.11.158.190:40427 was elected leader, sending a full tablet report...
I20260926 16:39:44.028648 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:44.020468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:36365 (local address 127.11.158.130:37779)
0926 16:39:44.020657 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:44.020661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:44.024350 (+ 3689us) server_negotiation.cc:407] Connection header received
0926 16:39:44.024564 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:44.024567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:44.024631 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:44.024683 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:44.025036 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.025766 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.027313 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.027888 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.027994 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:44.028060 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:44.028437 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 16:39:44.028520 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260926 16:39:44.028682 25165 consensus_queue.cc:260] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.029078 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:44.017377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:37779 (local address 127.11.158.131:36365)
0926 16:39:44.024251 (+ 6874us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:44.024267 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:44.024429 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:44.024824 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:44.024826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:44.024843 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:44.024915 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:44.024921 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.025904 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.025911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:44.027036 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:44.027046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.028163 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.028166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:44.028256 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:44.028285 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 16:39:44.028863 (+ 578us) client_negotiation.cc:240] Negotiation successful
0926 16:39:44.028945 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6810,"thread_start_us":84,"threads_started":1}
I20260926 16:39:44.029676 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec65c53004bb4656b04f42997be587c0" candidate_uuid: "62315196b3a54c83b90fb149df18181b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b8f3c9518d4c898efbb38b3358d402" is_pre_election: true
I20260926 16:39:44.029928 24951 raft_consensus.cc:2454] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62315196b3a54c83b90fb149df18181b in term 0.
I20260926 16:39:44.030324 25020 leader_election.cc:302] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62315196b3a54c83b90fb149df18181b, c4b8f3c9518d4c898efbb38b3358d402; no voters: 78aafdb9932b40e4933a0fe1f65926c1
I20260926 16:39:44.030769 25169 raft_consensus.cc:2781] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:39:44.030889 25169 raft_consensus.cc:482] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:39:44.030929 25169 raft_consensus.cc:3037] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:44.031138 25165 ts_tablet_manager.cc:1428] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260926 16:39:44.031455 24997 heartbeater.cc:503] Master 127.11.158.190:40427 was elected leader, sending a full tablet report...
I20260926 16:39:44.032228 25169 raft_consensus.cc:504] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.032617 25169 leader_election.cc:288] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [CANDIDATE]: Term 1 election: Requested vote from peers 78aafdb9932b40e4933a0fe1f65926c1 (127.11.158.129:33419), c4b8f3c9518d4c898efbb38b3358d402 (127.11.158.130:37779)
I20260926 16:39:44.034273 24804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec65c53004bb4656b04f42997be587c0" candidate_uuid: "62315196b3a54c83b90fb149df18181b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78aafdb9932b40e4933a0fe1f65926c1"
I20260926 16:39:44.034296 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec65c53004bb4656b04f42997be587c0" candidate_uuid: "62315196b3a54c83b90fb149df18181b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b8f3c9518d4c898efbb38b3358d402"
I20260926 16:39:44.034406 24951 raft_consensus.cc:3037] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:44.034423 24804 raft_consensus.cc:3037] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:44.035669 24804 raft_consensus.cc:2454] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62315196b3a54c83b90fb149df18181b in term 1.
I20260926 16:39:44.035691 24951 raft_consensus.cc:2454] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62315196b3a54c83b90fb149df18181b in term 1.
I20260926 16:39:44.036084 25020 leader_election.cc:302] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62315196b3a54c83b90fb149df18181b, c4b8f3c9518d4c898efbb38b3358d402; no voters:
I20260926 16:39:44.036475 25169 raft_consensus.cc:2781] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:39:44.036578 25169 raft_consensus.cc:686] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 1 LEADER]: Becoming Leader. State: Replica: 62315196b3a54c83b90fb149df18181b, State: Running, Role: LEADER
I20260926 16:39:44.036897 25169 consensus_queue.cc:237] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } }
I20260926 16:39:44.041339 24663 catalog_manager.cc:5495] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b reported cstate change: term changed from 0 to 1, leader changed from <none> to 62315196b3a54c83b90fb149df18181b (127.11.158.131). New cstate: current_term: 1 leader_uuid: "62315196b3a54c83b90fb149df18181b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62315196b3a54c83b90fb149df18181b" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 38405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 } health_report { overall_health: UNKNOWN } } }
I20260926 16:39:44.056046 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:44.051239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40514 (local address 127.11.158.131:38405)
0926 16:39:44.051359 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:44.051365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:44.051447 (+ 82us) server_negotiation.cc:407] Connection header received
0926 16:39:44.051625 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:44.051628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:44.051695 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:44.051806 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:39:44.052085 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.052974 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.054249 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:44.054593 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:44.054689 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:44.055046 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:39:44.055610 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:39:44.055731 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:44.055836 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 16:39:44.055900 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:39:44.062990 24951 raft_consensus.cc:1261] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 1 FOLLOWER]: Refusing update from remote peer 62315196b3a54c83b90fb149df18181b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:44.062983 24804 raft_consensus.cc:1261] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1 [term 1 FOLLOWER]: Refusing update from remote peer 62315196b3a54c83b90fb149df18181b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:44.063743 25169 consensus_queue.cc:1035] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [LEADER]: Connected to new peer: Peer: permanent_uuid: "78aafdb9932b40e4933a0fe1f65926c1" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 33419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:39:44.063928 25168 consensus_queue.cc:1035] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b8f3c9518d4c898efbb38b3358d402" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:39:44.075049 24863 tablet.cc:2369] T ec65c53004bb4656b04f42997be587c0 P 78aafdb9932b40e4933a0fe1f65926c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:39:44.078401 25179 mvcc.cc:205] Tried to move back new op lower bound from 7333645451514183680 to 7333645451417849856. Current Snapshot: MvccSnapshot[applied={T|T < 7333645451514183680}]
I20260926 16:39:44.092135 25177 mvcc.cc:205] Tried to move back new op lower bound from 7333645451514183680 to 7333645451417849856. Current Snapshot: MvccSnapshot[applied={T|T < 7333645451514183680 or (T in {7333645451514183680})}]
I20260926 16:39:44.092540 25178 mvcc.cc:205] Tried to move back new op lower bound from 7333645451514183680 to 7333645451417849856. Current Snapshot: MvccSnapshot[applied={T|T < 7333645451514183680 or (T in {7333645451514183680})}]
I20260926 16:39:44.093343 24663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:39:44.093508 24663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:39:44.096161 24663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 450ddfb0f14c4c87a8aa72932094fa2e: Sending DeleteTablet for 3 replicas of tablet ec65c53004bb4656b04f42997be587c0
I20260926 16:39:44.097193 24796 tablet_service.cc:1514] Processing DeleteTablet for tablet ec65c53004bb4656b04f42997be587c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:44 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41088
I20260926 16:39:44.098127 25057 tablet_service.cc:1514] Processing DeleteTablet for tablet ec65c53004bb4656b04f42997be587c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:44 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:40506
I20260926 16:39:44.099133 24931 tablet_service.cc:1514] Processing DeleteTablet for tablet ec65c53004bb4656b04f42997be587c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:44 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:39408
I20260926 16:39:44.100559 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24702
I20260926 16:39:44.105068 25189 tablet_replica.cc:330] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: stopping tablet replica
I20260926 16:39:44.105461 25189 raft_consensus.cc:2229] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:39:44.105751 25188 tablet_replica.cc:330] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: stopping tablet replica
I20260926 16:39:44.106153 25189 raft_consensus.cc:2258] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:39:44.106474 25188 raft_consensus.cc:2229] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:39:44.107091 25188 raft_consensus.cc:2258] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:39:44.108448 25189 ts_tablet_manager.cc:1905] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:44.113718 25188 ts_tablet_manager.cc:1905] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:44.116801 25189 ts_tablet_manager.cc:1918] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:44.122798 25189 log.cc:1196] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/ec65c53004bb4656b04f42997be587c0
I20260926 16:39:44.119793 25188 ts_tablet_manager.cc:1918] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:44.123405 25189 ts_tablet_manager.cc:1939] T ec65c53004bb4656b04f42997be587c0 P c4b8f3c9518d4c898efbb38b3358d402: Deleting consensus metadata
I20260926 16:39:44.123558 25188 log.cc:1196] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/ec65c53004bb4656b04f42997be587c0
I20260926 16:39:44.123940 25188 ts_tablet_manager.cc:1939] T ec65c53004bb4656b04f42997be587c0 P 62315196b3a54c83b90fb149df18181b: Deleting consensus metadata
I20260926 16:39:44.137455 24648 catalog_manager.cc:4841] TS 62315196b3a54c83b90fb149df18181b (127.11.158.131:38405): tablet ec65c53004bb4656b04f42997be587c0 (table test-table [id=ab26a0b740a24a2fb26e311e8d97f816]) successfully deleted
I20260926 16:39:44.138684 24651 catalog_manager.cc:4841] TS c4b8f3c9518d4c898efbb38b3358d402 (127.11.158.130:37779): tablet ec65c53004bb4656b04f42997be587c0 (table test-table [id=ab26a0b740a24a2fb26e311e8d97f816]) successfully deleted
W20260926 16:39:44.139191 24651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:33419 (error 108)
I20260926 16:39:44.140164 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24866
I20260926 16:39:44.146525 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:44.146167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:33419 (local address 127.0.0.1:41098)
0926 16:39:44.146358 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:44.146432 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260926 16:39:44.146859 24651 catalog_manager.cc:4584] TS 78aafdb9932b40e4933a0fe1f65926c1 (127.11.158.129:33419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec65c53004bb4656b04f42997be587c0: Network error: Client connection negotiation failed: client connection to 127.11.158.129:33419: connect: Connection refused (error 111)
I20260926 16:39:44.158429 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24999
I20260926 16:39:44.167507 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 24631
I20260926 16:39:44.173581 11898 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 16:39:44.508401 11898 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 16:39:44.508522 11898 mini_ranger.cc:66] Stopping Ranger...
I20260926 16:39:44.853413 11898 mini_ranger.cc:68] Stopped Ranger
2026-09-26 16:39:44.853 UTC [23698] LOG: received smart shutdown request
2026-09-26 16:39:44.857 UTC [23698] LOG: background worker "logical replication launcher" (PID 23707) exited with exit code 1
2026-09-26 16:39:44.857 UTC [23702] LOG: shutting down
2026-09-26 16:39:45.053 UTC [23698] LOG: database system is shut down
2026-09-26T16:39:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49906 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:39:45 dist-test-slave-p39x krb5kdc[25199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:39:45 dist-test-slave-p39x krb5kdc[25199](info): set up 2 sockets
Sep 26 16:39:45 dist-test-slave-p39x krb5kdc[25199](info): commencing operation
krb5kdc: starting...
W20260926 16:39:46.217048 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:39:46.344532 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33683
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:33661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33683,127.11.158.189:41465,127.11.158.188:33287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:46.457319 25212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:46.457633 25212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:46.457733 25212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:46.462070 25212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:46.462178 25212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:46.462221 25212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:46.462260 25212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:46.462299 25212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:46.467763 25212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:33661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33683,127.11.158.189:41465,127.11.158.188:33287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:46.468854 25212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:46.469867 25212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:46.478271 25220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:46.478312 25218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:46.478283 25217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:46.478611 25212 server_base.cc:1034] running on GCE node
I20260926 16:39:46.479271 25212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:46.480496 25212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:46.481699 25212 hybrid_clock.cc:648] HybridClock initialized: now 1790440786481687 us; error 37 us; skew 500 ppm
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:46.484405 25212 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:39:46.485280 25212 webserver.cc:460] Webserver started at http://127.11.158.190:41617/ using document root <none> and password file <none>
I20260926 16:39:46.485844 25212 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:46.485935 25212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:46.486167 25212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:46.488317 25212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "ec1165e48414464992994ae5862e488e"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "dceae83ed2f30cf3dcfdb055d647eb39"
server_key_iv: "2a3ab7c13f3cbed691352df422827de2"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.488966 25212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "ec1165e48414464992994ae5862e488e"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "dceae83ed2f30cf3dcfdb055d647eb39"
server_key_iv: "2a3ab7c13f3cbed691352df422827de2"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.492683 25212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:39:46.495056 25227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:46.496271 25212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:39:46.496460 25212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "ec1165e48414464992994ae5862e488e"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "dceae83ed2f30cf3dcfdb055d647eb39"
server_key_iv: "2a3ab7c13f3cbed691352df422827de2"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.496582 25212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:46.527241 25212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:46.530918 25212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:46.531133 25212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:46.539497 25212 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:33683
I20260926 16:39:46.539829 25279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:33683 every 8 connection(s)
I20260926 16:39:46.540594 25212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:39:46.541306 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25212
I20260926 16:39:46.541445 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:39:46.541719 11898 external_mini_cluster.cc:1428] Setting key f6c0c214f8d926d9f6d79a7ffc6dc113
I20260926 16:39:46.543941 25280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:46.547828 25280 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.158.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.189@KRBTEST.COM" created.
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:46.565199 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.550330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33683 (local address 127.0.0.1:52294)
0926 16:39:46.550933 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.550982 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:46.551829 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:46.552636 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:46.552641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:46.552962 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:46.553317 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.553330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.554666 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.554670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.555959 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.555973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.556727 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.556732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.556886 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.557565 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:46.557591 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:46.559379 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:46.561669 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.561679 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.561690 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.561919 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.562299 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.562302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.562304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.562415 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.563175 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:46.563182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:46.563413 (+ 231us) client_negotiation.cc:769] Sending connection context
0926 16:39:46.563699 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 16:39:46.564138 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":198,"thread_start_us":150,"threads_started":1}
I20260926 16:39:46.565205 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.550330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.11.158.190:33683)
0926 16:39:46.550879 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:46.550896 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:46.551084 (+ 188us) server_negotiation.cc:407] Connection header received
0926 16:39:46.552166 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:46.552172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:46.552428 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:46.552624 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:46.553487 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.554537 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.556099 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.556558 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.556682 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.559571 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:46.559577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:46.559588 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:46.559610 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:46.561527 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.562054 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.562092 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.562093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.562196 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.562540 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.562543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.562544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.563055 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:46.563200 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:46.563770 (+ 570us) server_negotiation.cc:299] Negotiation successful
0926 16:39:46.564088 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":423,"spinlock_wait_cycles":59136,"thread_start_us":162,"threads_started":1}
I20260926 16:39:46.567857 25280 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:39:46.568809 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.568508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42092)
0926 16:39:46.568653 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.568735 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 16:39:46.569001 25228 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
I20260926 16:39:46.569972 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.569724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42102)
0926 16:39:46.569850 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.569905 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 16:39:46.570142 25280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:41465: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab.
Entry for principal kudu/127.11.158.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab.
I20260926 16:39:46.571602 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:41465
--webserver_interface=127.11.158.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab
--principal=kudu/127.11.158.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:33661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33683,127.11.158.189:41465,127.11.158.188:33287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:39:46.619415 25280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } attempt: 1
I20260926 16:39:46.620487 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.620113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42114)
0926 16:39:46.620278 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.620404 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 16:39:46.621357 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.621113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42130)
0926 16:39:46.621242 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.621300 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 16:39:46.621502 25280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:41465: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
W20260926 16:39:46.684445 25287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:46.684810 25287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:46.684916 25287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:46.689337 25287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:46.689427 25287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:46.689463 25287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:46.689530 25287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:46.689569 25287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:46.689687 25280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } attempt: 2
I20260926 16:39:46.690533 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.690231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42146)
0926 16:39:46.690376 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.690456 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 16:39:46.691296 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.691077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42160)
0926 16:39:46.691188 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.691237 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:39:46.691449 25280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:41465: Network error: Client connection negotiation failed: client connection to 127.11.158.189:41465: connect: Connection refused (error 111)
I20260926 16:39:46.695840 25287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:33661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33683,127.11.158.189:41465,127.11.158.188:33287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab
--principal=kudu/127.11.158.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:41465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.158.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:46.697156 25287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:46.698240 25287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:46.707134 25292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:46.707134 25295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:46.707155 25293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:46.707456 25287 server_base.cc:1034] running on GCE node
I20260926 16:39:46.708081 25287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:46.709323 25287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:46.710511 25287 hybrid_clock.cc:648] HybridClock initialized: now 1790440786710473 us; error 52 us; skew 500 ppm
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:46.713455 25287 init.cc:377] Logged in from keytab as kudu/127.11.158.189@KRBTEST.COM (short username kudu)
I20260926 16:39:46.714358 25287 webserver.cc:460] Webserver started at http://127.11.158.189:43879/ using document root <none> and password file <none>
I20260926 16:39:46.714910 25287 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:46.714969 25287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:46.715139 25287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:46.717310 25287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data/instance:
uuid: "3869e00ca9be43308567c4e5f7441fe5"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "3f36ae3ed807eab8b850a400c222f47b"
server_key_iv: "86f19987326f5b871222a79fb965c609"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.717875 25287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/wal/instance:
uuid: "3869e00ca9be43308567c4e5f7441fe5"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "3f36ae3ed807eab8b850a400c222f47b"
server_key_iv: "86f19987326f5b871222a79fb965c609"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.721711 25287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:39:46.724431 25302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:46.725766 25287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:46.725940 25287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/wal
uuid: "3869e00ca9be43308567c4e5f7441fe5"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "3f36ae3ed807eab8b850a400c222f47b"
server_key_iv: "86f19987326f5b871222a79fb965c609"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.726078 25287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:46.766880 25287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:46.771034 25287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:46.771270 25287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:46.778573 25287 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.189:41465
I20260926 16:39:46.778924 25354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.189:41465 every 8 connection(s)
I20260926 16:39:46.779659 25287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
I20260926 16:39:46.782629 25280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } attempt: 3
I20260926 16:39:46.782992 25355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:46.787495 25355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:39:46.788935 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25287
I20260926 16:39:46.789076 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-1/wal/instance
I20260926 16:39:46.789408 11898 external_mini_cluster.cc:1428] Setting key 151c8414f22dc092927a8e2ae808de51
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:39:46.797219 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.783214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42170)
0926 16:39:46.783371 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.783390 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:46.783509 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:46.785942 (+ 2433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:46.785945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:46.786157 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:46.786292 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.786301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.787776 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.787783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.788439 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.788452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.789309 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.789313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.789427 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.790039 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:46.790059 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:46.791732 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:46.794549 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.794555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.794560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.794971 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.795454 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.795457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.795459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.795579 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.796572 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:46.796575 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:46.796692 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 16:39:46.796820 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 16:39:46.796982 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:46.798559 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.784043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.11.158.189:41465)
0926 16:39:46.784563 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:46.784578 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:46.784615 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:39:46.785293 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:46.785310 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:46.785646 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:46.786018 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:46.786477 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.787609 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.788597 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.789174 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.789347 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.791928 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:46.791934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:46.791946 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:46.791979 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:46.794416 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.795123 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.795132 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.795140 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.795340 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.795718 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.795724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.795728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.796434 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:46.796596 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:46.797025 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 16:39:46.797370 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":155,"threads_started":1}
WARNING: no policy specified for kudu/127.11.158.188@KRBTEST.COM; defaulting to no policy
I20260926 16:39:46.801028 25280 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.188@KRBTEST.COM" created.
I20260926 16:39:46.801877 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.801644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55200)
0926 16:39:46.801754 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.801807 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 16:39:46.802563 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.802318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55214)
0926 16:39:46.802457 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.802503 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 16:39:46.802604 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.790750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.11.158.190:33683)
0926 16:39:46.790924 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:46.790930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:46.790948 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:39:46.790994 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:46.791000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:46.791054 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:46.791144 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:46.792057 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.793051 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.794634 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.795523 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.795640 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.798266 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:46.798270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:46.798273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:46.798299 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:46.800283 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.800798 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.800802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.800806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.800901 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.801288 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.801291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.801293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.801530 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:46.801636 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:46.802167 (+ 531us) server_negotiation.cc:299] Negotiation successful
0926 16:39:46.802305 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 16:39:46.802791 25280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33287: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
I20260926 16:39:46.803346 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.789489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33683 (local address 127.0.0.1:52300)
0926 16:39:46.790297 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.790354 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:46.790466 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:46.791324 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:46.791329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:46.791738 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:46.791931 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.791938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.793469 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.793476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.794540 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.794547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.795645 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.795649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.795776 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.796395 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:46.796410 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:46.798124 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:46.800415 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.800424 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.800430 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.800705 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.801023 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.801026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.801028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.801147 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.801694 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:46.801703 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:46.801848 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 16:39:46.802808 (+ 960us) client_negotiation.cc:240] Negotiation successful
0926 16:39:46.803050 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":477,"threads_started":1}
I20260926 16:39:46.804540 25355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:39:46.813731 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.805157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.11.158.189:41465)
0926 16:39:46.805287 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:46.805294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:46.805311 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:39:46.805357 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:46.805362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:46.805412 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:46.805514 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:46.805934 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.806862 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.807633 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.808093 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.808206 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.810438 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:46.810442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:46.810444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:46.810473 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:46.811921 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.812439 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.812442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.812443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.812546 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:46.812833 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:46.812839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:46.812843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:46.813080 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:46.813163 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:46.813402 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 16:39:46.813538 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":47}
I20260926 16:39:46.813735 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.805094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42184)
0926 16:39:46.805206 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.805221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:46.805300 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:46.805537 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:46.805539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:46.805677 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:46.805771 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.805776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.806991 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.807000 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.807512 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:46.807521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:46.808228 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:46.808231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:46.808304 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:46.808850 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:46.808864 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:46.810302 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:46.812074 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.812079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.812081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.812292 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.812656 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:46.812659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:46.812660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:46.812736 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:46.813173 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:46.813176 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:46.813258 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 16:39:46.813352 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 16:39:46.813478 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 16:39:46.814885 25355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:39:46.815680 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.815389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55230)
0926 16:39:46.815529 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.815615 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260926 16:39:46.816460 25305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
I20260926 16:39:46.817379 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.817174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55240)
0926 16:39:46.817273 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.817320 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 16:39:46.817559 25355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33287: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab.
Entry for principal kudu/127.11.158.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab.
I20260926 16:39:46.820448 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:33287
--webserver_interface=127.11.158.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab
--principal=kudu/127.11.158.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:33661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33683,127.11.158.189:41465,127.11.158.188:33287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:39:46.833999 25280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } attempt: 1
I20260926 16:39:46.834944 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.834602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55256)
0926 16:39:46.834788 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.834857 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 16:39:46.835731 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.835513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55258)
0926 16:39:46.835616 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.835673 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 16:39:46.835886 25280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33287: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
I20260926 16:39:46.866864 25355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } attempt: 1
I20260926 16:39:46.867854 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.867509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55274)
0926 16:39:46.867706 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.867773 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 16:39:46.868675 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.868450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55276)
0926 16:39:46.868558 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.868614 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 16:39:46.868808 25355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33287: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
I20260926 16:39:46.911067 25280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } attempt: 2
I20260926 16:39:46.911932 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.911616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55286)
0926 16:39:46.911790 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.911853 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 16:39:46.912732 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.912505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55290)
0926 16:39:46.912624 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.912678 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 16:39:46.912856 25280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33287: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
I20260926 16:39:46.936975 25355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } attempt: 2
W20260926 16:39:46.937233 25362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:46.937544 25362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:46.937670 25362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 16:39:46.937853 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.937494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55306)
0926 16:39:46.937692 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.937767 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 16:39:46.938551 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:46.938332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55310)
0926 16:39:46.938440 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:46.938489 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:39:46.938702 25355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33287: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33287: connect: Connection refused (error 111)
W20260926 16:39:46.942154 25362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:46.942251 25362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:46.942298 25362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:46.942334 25362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:46.942370 25362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:46.947911 25362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:33661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33683,127.11.158.189:41465,127.11.158.188:33287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab
--principal=kudu/127.11.158.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:33287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.158.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:46.949137 25362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:46.950209 25362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:46.958904 25367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:46.958947 25368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:46.958904 25370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:46.959388 25362 server_base.cc:1034] running on GCE node
I20260926 16:39:46.960045 25362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:46.961275 25362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:46.962440 25362 hybrid_clock.cc:648] HybridClock initialized: now 1790440786962395 us; error 59 us; skew 500 ppm
Sep 26 16:39:46 dist-test-slave-p39x krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:46.965306 25362 init.cc:377] Logged in from keytab as kudu/127.11.158.188@KRBTEST.COM (short username kudu)
I20260926 16:39:46.966214 25362 webserver.cc:460] Webserver started at http://127.11.158.188:41543/ using document root <none> and password file <none>
I20260926 16:39:46.966791 25362 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:46.966884 25362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:46.967128 25362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:46.969313 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data/instance:
uuid: "4a9131be273d455fa5749f1523b9f02b"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "141873945b319ae42d596e5089d6de43"
server_key_iv: "024777f23c6ab1534232dcab7d2249a6"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.969885 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/wal/instance:
uuid: "4a9131be273d455fa5749f1523b9f02b"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "141873945b319ae42d596e5089d6de43"
server_key_iv: "024777f23c6ab1534232dcab7d2249a6"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.973723 25362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:39:46.976301 25377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:46.977658 25362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:39:46.977816 25362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/wal
uuid: "4a9131be273d455fa5749f1523b9f02b"
format_stamp: "Formatted at 2026-09-26 16:39:46 on dist-test-slave-p39x"
server_key: "141873945b319ae42d596e5089d6de43"
server_key_iv: "024777f23c6ab1534232dcab7d2249a6"
server_key_version: "encryptionkey@0"
I20260926 16:39:46.977943 25362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:46.990402 25362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:46.993352 25362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:46.993558 25362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:47.000633 25362 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.188:33287
I20260926 16:39:47.000972 25429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.188:33287 every 8 connection(s)
I20260926 16:39:47.001698 25362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
I20260926 16:39:47.004550 25430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:47.007138 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25362
I20260926 16:39:47.007277 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440622937842-11898-0/minicluster-data/master-2/wal/instance
I20260926 16:39:47.007583 11898 external_mini_cluster.cc:1428] Setting key 3e3259be711bb0ce0773447aa3fcf469
I20260926 16:39:47.009017 25430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:39:47.012013 25280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } attempt: 3
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:47.022310 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.012436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55326)
0926 16:39:47.012556 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:47.012571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:47.012652 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:47.013017 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:47.013020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:47.013196 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:47.013285 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.013291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.014308 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.014312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.015045 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.015052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.015645 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.015649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.015751 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.016176 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:47.016192 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:47.017532 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:47.020029 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.020033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.020036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.020445 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.020845 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.020849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.020851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.020951 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.021703 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:47.021705 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:47.021806 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 16:39:47.021967 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 16:39:47.022121 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":1}
I20260926 16:39:47.022992 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.010777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.11.158.190:33683)
0926 16:39:47.010895 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.010900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.011218 (+ 318us) server_negotiation.cc:407] Connection header received
0926 16:39:47.011436 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.011440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.011485 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.011553 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.012554 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.013252 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.014621 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.014962 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.015066 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.018937 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.018941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.018943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.018970 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.020918 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.021452 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.021455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.021457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.021539 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.021833 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.021836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.021837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.022047 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.022143 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.022734 (+ 591us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.022850 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:39:47.023991 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.012481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.11.158.188:33287)
0926 16:39:47.012795 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.012804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.012817 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:39:47.012848 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.012850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.012886 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.012971 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.013403 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.014177 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.015159 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.015524 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.015635 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.017660 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.017665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.017674 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.017694 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.019901 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.020573 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.020580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.020587 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.020729 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.021082 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.021086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.021090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.021615 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.021742 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.022619 (+ 877us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.023080 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"spinlock_wait_cycles":12544,"thread_start_us":120,"threads_started":1}
I20260926 16:39:47.023993 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.008927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.11.158.188:33287)
0926 16:39:47.009364 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.009404 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.009448 (+ 44us) server_negotiation.cc:407] Connection header received
0926 16:39:47.010096 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.010112 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.010513 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.010861 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.011884 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.013480 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.014312 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.014806 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.014965 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.018307 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.018312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.018316 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.018350 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.020986 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.021488 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.021491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.021493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.021567 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.021892 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.021894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.021896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.022058 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.022239 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.022906 (+ 667us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.023234 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":286,"thread_start_us":142,"threads_started":1}
I20260926 16:39:47.023993 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.010685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:33683 (local address 127.0.0.1:52316)
0926 16:39:47.011079 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:47.011135 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:47.011313 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:47.011644 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:47.011650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:47.012143 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:47.012397 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.012412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.013355 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.013361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.014513 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.014520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.015177 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.015182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.015314 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.016016 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:47.016038 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:47.018794 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:47.021033 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.021043 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.021049 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.021341 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.021654 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.021657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.021662 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.021754 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.022196 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:47.022204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:47.022350 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 16:39:47.023176 (+ 826us) client_negotiation.cc:240] Negotiation successful
0926 16:39:47.023343 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
I20260926 16:39:47.025944 25430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:39:47.028657 25280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e: Bootstrap starting.
I20260926 16:39:47.030196 25355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } attempt: 3
I20260926 16:39:47.031602 25280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e: Neither blocks nor log segments found. Creating new log.
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:39:47.032537 25280 log.cc:824] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:47.034988 25280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e: No bootstrap required, opened a new log
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:39:47.036948 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.026582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.11.158.189:41465)
0926 16:39:47.026733 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.026738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.027157 (+ 419us) server_negotiation.cc:407] Connection header received
0926 16:39:47.027252 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.027255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.027312 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.027404 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.027907 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.028692 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.029682 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.030012 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.030110 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.032790 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.032794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.032797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.032824 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.034717 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.035257 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.035263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.035268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.035367 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.035824 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.035827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.035829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.036054 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.036141 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.036589 (+ 448us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.036764 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:39:47.037513 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.026471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:41465 (local address 127.0.0.1:42186)
0926 16:39:47.027062 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:47.027086 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:47.027158 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:47.027440 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:47.027443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:47.027616 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:47.027768 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.027776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.028822 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.028825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.029565 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.029576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.030127 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.030131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.030231 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.031068 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:47.031081 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:47.032630 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:47.034847 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.034852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.034857 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.035140 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.035612 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.035615 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.035618 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.035708 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.036215 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:47.036221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:47.036379 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 16:39:47.036709 (+ 330us) client_negotiation.cc:240] Negotiation successful
0926 16:39:47.037312 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515}
I20260926 16:39:47.038254 25430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:39:47.038842 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.026805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.11.158.190:33683)
0926 16:39:47.026953 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.026957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.026969 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:39:47.027087 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.027092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.027143 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.027213 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.028083 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.028665 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.029491 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.029782 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.029904 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.034037 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.034043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.034046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.034076 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.036020 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.036595 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.036599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.036601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.036686 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.036993 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.036996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.036998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.037477 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.037674 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.038517 (+ 843us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.038665 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:39:47.039733 25280 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.039983 25280 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:47.040062 25280 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec1165e48414464992994ae5862e488e, State: Initialized, Role: FOLLOWER
I20260926 16:39:47.040689 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.030586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55334)
0926 16:39:47.030684 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:47.030699 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:47.030774 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:47.032037 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:47.032040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:47.032229 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:47.032370 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.032380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.033505 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.033512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.034272 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.034283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.034887 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.034891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.034970 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.035329 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:47.035347 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:47.036795 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:47.038708 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.038711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.038713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.038955 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.039357 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.039359 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.039361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.039449 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.040106 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:47.040110 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:47.040207 (+ 97us) client_negotiation.cc:769] Sending connection context
0926 16:39:47.040290 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:39:47.040491 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 16:39:47.041288 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.030756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.11.158.188:33287)
0926 16:39:47.031811 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.031815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.031828 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:39:47.031861 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.031864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.031902 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.031991 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.032515 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.033357 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.034419 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.034755 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.035157 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.036939 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.036943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.036946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.036972 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.038583 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.039076 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.039079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.039081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.039159 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.039787 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.039790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.039792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.039982 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.040932 (+ 950us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.041014 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.041138 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970}
I20260926 16:39:47.041577 25280 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.043426 25280 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:39:47.044570 25355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: Bootstrap starting.
Sep 26 16:39:47 dist-test-slave-p39x krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:39:47.047118 25438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.047258 25438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:39:47.047534 25355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:47.048259 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.039160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55336 (local address 127.11.158.188:33287)
0926 16:39:47.039295 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.039299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.039529 (+ 230us) server_negotiation.cc:407] Connection header received
0926 16:39:47.039564 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.039566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.039606 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.039679 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.040128 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.040909 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.041980 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.042321 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.042405 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.044757 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.044762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.044766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.044792 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.046492 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.046952 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.046955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.046957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.047023 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.047315 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.047318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.047320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.047527 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.047614 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.047882 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.048085 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":68}
I20260926 16:39:47.048439 25355 log.cc:824] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:47.049358 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.038826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33287 (local address 127.0.0.1:55336)
0926 16:39:47.039280 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:47.039293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:47.039383 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:47.039725 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:47.039728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:47.039898 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:47.039997 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.040004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.041025 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.041028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.041867 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:47.041877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.042501 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.042504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:47.042581 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.043058 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:47.043082 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:47.044594 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:47.046612 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.046615 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.046617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.046855 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.047131 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:47.047134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:47.047135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:47.047213 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:47.047647 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:47.047649 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:47.047727 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 16:39:47.047840 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 16:39:47.047996 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":116,"threads_started":1}
I20260926 16:39:47.052287 25430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: Bootstrap starting.
W20260926 16:39:47.052834 25449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:39:47.052904 25449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:39:47.050716 25212 master_runner.cc:186] Error getting master registration for 127.11.158.189:41465: OK, instance_id { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" instance_seqno: 1790440786776539 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:39:47.053895 25212 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:39:47.055115 25430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:47.055521 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.040119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.11.158.189:41465)
0926 16:39:47.040228 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.040232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.040456 (+ 224us) server_negotiation.cc:407] Connection header received
0926 16:39:47.040624 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.040627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.040665 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.040710 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.041513 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.042224 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.043625 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.043963 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.044044 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.047676 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.047680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.047683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.047713 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.049648 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.054211 (+ 4563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.054215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.054217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.054328 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.054654 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.054658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.054660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.054861 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.055082 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.055226 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.055345 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:39:47.056015 25430 log.cc:824] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:47.058161 25355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: No bootstrap required, opened a new log
I20260926 16:39:47.058964 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:39:47.062094 25355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.062479 25355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:47.062601 25355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3869e00ca9be43308567c4e5f7441fe5, State: Initialized, Role: FOLLOWER
I20260926 16:39:47.063325 25355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.065076 25355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:47.073781 25430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: No bootstrap required, opened a new log
I20260926 16:39:47.075002 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.061920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.11.158.190:33683)
0926 16:39:47.062162 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.062167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.062183 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:39:47.062228 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.062231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.062279 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.062370 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.063306 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.064135 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.065399 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.065842 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.065935 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.070641 (+ 4706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.070645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.070648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.070677 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.072255 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.072783 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.072787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.072789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.072875 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.073174 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.073178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.073180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.073513 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.073666 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.074701 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.074838 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260926 16:39:47.074558 25287 master_runner.cc:186] Error getting master registration for 127.11.158.188:33287: OK, instance_id { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" instance_seqno: 1790440786998538 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:39:47.079504 25430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.079818 25430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:47.079879 25430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a9131be273d455fa5749f1523b9f02b, State: Initialized, Role: FOLLOWER
I20260926 16:39:47.080526 25430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.079262 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.062023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42200 (local address 127.11.158.189:41465)
0926 16:39:47.062536 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.062541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.063161 (+ 620us) server_negotiation.cc:407] Connection header received
0926 16:39:47.063403 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.063407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.063454 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.063509 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.066655 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.067421 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.069697 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.070040 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.070127 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.075456 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.075461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.075464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.075491 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.077411 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.077905 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.077908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.077910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.077994 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.078303 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.078306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.078308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.078524 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.078632 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.078962 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.079098 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428}
I20260926 16:39:47.082163 25430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:47.082708 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:47.061697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.11.158.188:33287)
0926 16:39:47.061815 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:47.061821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:47.062936 (+ 1115us) server_negotiation.cc:407] Connection header received
0926 16:39:47.063651 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:47.063657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:47.063832 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:47.063895 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:47.065683 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.066475 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.067685 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:47.068029 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:47.068124 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:47.069912 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:47.069916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:47.069919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:47.069949 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:47.080653 (+ 10704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.081235 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.081239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.081241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.081331 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:47.081617 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:47.081621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:47.081623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:47.081840 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:47.081966 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:47.082427 (+ 461us) server_negotiation.cc:299] Negotiation successful
0926 16:39:47.082556 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:39:47.084771 25287 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:39:47.085376 25472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.088023 25472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [sys.catalog]: This master's current role is: FOLLOWER
W20260926 16:39:47.088088 25471 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:39:47.094339 25484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 16:39:47.090945 25460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.094818 25460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 16:39:47.094952 25471 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 16:39:47.095077 25484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:39:47.096262 25362 master_runner.cc:263] No leader master found from master 4a9131be273d455fa5749f1523b9f02b
I20260926 16:39:47.096324 25362 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:39:47.211028 25438 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:39:47.211261 25438 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.212409 25438 leader_election.cc:288] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3869e00ca9be43308567c4e5f7441fe5 (127.11.158.189:41465), 4a9131be273d455fa5749f1523b9f02b (127.11.158.188:33287)
I20260926 16:39:47.213449 25329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec1165e48414464992994ae5862e488e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3869e00ca9be43308567c4e5f7441fe5" is_pre_election: true
I20260926 16:39:47.213801 25329 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec1165e48414464992994ae5862e488e in term 0.
I20260926 16:39:47.214403 25228 leader_election.cc:302] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3869e00ca9be43308567c4e5f7441fe5, ec1165e48414464992994ae5862e488e; no voters:
I20260926 16:39:47.214802 25438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:39:47.214883 25438 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:39:47.214928 25438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:47.216545 25438 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.217221 25438 leader_election.cc:288] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [CANDIDATE]: Term 1 election: Requested vote from peers 3869e00ca9be43308567c4e5f7441fe5 (127.11.158.189:41465), 4a9131be273d455fa5749f1523b9f02b (127.11.158.188:33287)
I20260926 16:39:47.217758 25329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec1165e48414464992994ae5862e488e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3869e00ca9be43308567c4e5f7441fe5"
I20260926 16:39:47.217902 25329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:47.219249 25329 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec1165e48414464992994ae5862e488e in term 1.
I20260926 16:39:47.220046 25404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec1165e48414464992994ae5862e488e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a9131be273d455fa5749f1523b9f02b" is_pre_election: true
I20260926 16:39:47.220412 25404 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec1165e48414464992994ae5862e488e in term 0.
I20260926 16:39:47.220976 25404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ec1165e48414464992994ae5862e488e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a9131be273d455fa5749f1523b9f02b"
I20260926 16:39:47.221105 25404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:47.222543 25404 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec1165e48414464992994ae5862e488e in term 1.
I20260926 16:39:47.219699 25228 leader_election.cc:302] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3869e00ca9be43308567c4e5f7441fe5, ec1165e48414464992994ae5862e488e; no voters:
I20260926 16:39:47.223853 25438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:39:47.224136 25438 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [term 1 LEADER]: Becoming Leader. State: Replica: ec1165e48414464992994ae5862e488e, State: Running, Role: LEADER
I20260926 16:39:47.224692 25438 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } }
I20260926 16:39:47.230207 25438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1165e48414464992994ae5862e488e. Latest consensus state: current_term: 1 leader_uuid: "ec1165e48414464992994ae5862e488e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.230428 25438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:47.231010 25488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:39:47.232133 25488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:39:47.240001 25404 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [term 1 FOLLOWER]: Refusing update from remote peer ec1165e48414464992994ae5862e488e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:47.240952 25486 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:39:47.244957 25383 tablet.cc:2369] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:39:47.246529 25472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1165e48414464992994ae5862e488e. Latest consensus state: current_term: 1 leader_uuid: "ec1165e48414464992994ae5862e488e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.246709 25472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:39:47.248095 25329 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [term 1 FOLLOWER]: Refusing update from remote peer ec1165e48414464992994ae5862e488e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:47.251762 25438 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:39:47.256678 25460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1165e48414464992994ae5862e488e. Latest consensus state: current_term: 1 leader_uuid: "ec1165e48414464992994ae5862e488e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.256843 25460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:39:47.261639 25486 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec1165e48414464992994ae5862e488e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.261797 25486 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:47.263168 25460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec1165e48414464992994ae5862e488e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.263298 25460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:39:47.263839 25489 mvcc.cc:205] Tried to move back new op lower bound from 7333645464526262272 to 7333645464486465536. Current Snapshot: MvccSnapshot[applied={T|T < 7333645464526262272}]
I20260926 16:39:47.265317 25492 mvcc.cc:205] Tried to move back new op lower bound from 7333645464526262272 to 7333645464486465536. Current Snapshot: MvccSnapshot[applied={T|T < 7333645464526262272}]
I20260926 16:39:47.274456 25438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec1165e48414464992994ae5862e488e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.274647 25438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:47.275954 25472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec1165e48414464992994ae5862e488e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1165e48414464992994ae5862e488e" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33683 } } peers { permanent_uuid: "3869e00ca9be43308567c4e5f7441fe5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 41465 } } peers { permanent_uuid: "4a9131be273d455fa5749f1523b9f02b" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33287 } } }
I20260926 16:39:47.276086 25472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:39:47.277271 25488 catalog_manager.cc:1347] Generated new cluster ID: ad8b05a855da439b9494ee73f809e42d
I20260926 16:39:47.277340 25488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:39:47.279842 25490 mvcc.cc:205] Tried to move back new op lower bound from 7333645464526262272 to 7333645464486465536. Current Snapshot: MvccSnapshot[applied={T|T < 7333645464526262272}]
I20260926 16:39:47.306978 25488 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:39:47.308305 25488 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:39:47.320180 25488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec1165e48414464992994ae5862e488e: Generated new TSK 0
I20260926 16:39:47.321255 25488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:39:48.096297 25471 catalog_manager.cc:1259] Loaded cluster ID: ad8b05a855da439b9494ee73f809e42d
I20260926 16:39:48.096297 25484 catalog_manager.cc:1259] Loaded cluster ID: ad8b05a855da439b9494ee73f809e42d
I20260926 16:39:48.096441 25471 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: loading cluster ID for follower catalog manager: success
I20260926 16:39:48.096444 25484 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: loading cluster ID for follower catalog manager: success
I20260926 16:39:48.098043 25484 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: acquiring CA information for follower catalog manager: success
I20260926 16:39:48.098050 25471 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: acquiring CA information for follower catalog manager: success
I20260926 16:39:48.099157 25471 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3869e00ca9be43308567c4e5f7441fe5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:39:48.099157 25484 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4a9131be273d455fa5749f1523b9f02b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:39:48.385689 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25212
I20260926 16:39:48.392800 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25287
I20260926 16:39:48.398770 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25362
2026-09-26T16:39:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3231 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:39:48 dist-test-slave-p39x krb5kdc[25503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:39:48 dist-test-slave-p39x krb5kdc[25503](info): set up 2 sockets
Sep 26 16:39:48 dist-test-slave-p39x krb5kdc[25503](info): commencing operation
krb5kdc: starting...
W20260926 16:39:49.461362 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:39:49 dist-test-slave-p39x krb5kdc[25503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:39:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:39:49.588274 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34059
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:34059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:49.703243 25516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:49.703622 25516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:49.703711 25516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:49.708168 25516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:49.708297 25516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:49.708334 25516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:49.708388 25516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:49.708427 25516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:49.714375 25516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:34059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:49.715579 25516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:49.716820 25516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:49.725885 25524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:49.725879 25522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:49.725920 25516 server_base.cc:1034] running on GCE node
W20260926 16:39:49.725941 25521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:49.726773 25516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:49.727900 25516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:49.729076 25516 hybrid_clock.cc:648] HybridClock initialized: now 1790440789729052 us; error 38 us; skew 500 ppm
Sep 26 16:39:49 dist-test-slave-p39x krb5kdc[25503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440789, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:49.731777 25516 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:39:49.732752 25516 webserver.cc:460] Webserver started at http://127.11.158.190:35221/ using document root <none> and password file <none>
I20260926 16:39:49.733330 25516 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:49.733418 25516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:49.733646 25516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:49.735711 25516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "e572c5a9aff84d17baf46b34731d881d"
format_stamp: "Formatted at 2026-09-26 16:39:49 on dist-test-slave-p39x"
server_key: "8c751402a46415474d0fdcc614283cd5"
server_key_iv: "8f71b9f24e219984da6a189423381df0"
server_key_version: "encryptionkey@0"
I20260926 16:39:49.736267 25516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "e572c5a9aff84d17baf46b34731d881d"
format_stamp: "Formatted at 2026-09-26 16:39:49 on dist-test-slave-p39x"
server_key: "8c751402a46415474d0fdcc614283cd5"
server_key_iv: "8f71b9f24e219984da6a189423381df0"
server_key_version: "encryptionkey@0"
I20260926 16:39:49.739852 25516 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:49.742324 25531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:49.743506 25516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:39:49.743641 25516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "e572c5a9aff84d17baf46b34731d881d"
format_stamp: "Formatted at 2026-09-26 16:39:49 on dist-test-slave-p39x"
server_key: "8c751402a46415474d0fdcc614283cd5"
server_key_iv: "8f71b9f24e219984da6a189423381df0"
server_key_version: "encryptionkey@0"
I20260926 16:39:49.743753 25516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:49.754827 25516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:49.757843 25516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:49.758051 25516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:49.765313 25516 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:34059
I20260926 16:39:49.765375 25583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:34059 every 8 connection(s)
I20260926 16:39:49.766312 25516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:39:49.769146 25584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:49.775086 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25516
I20260926 16:39:49.775100 25584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d: Bootstrap starting.
I20260926 16:39:49.775233 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:39:49.775550 11898 external_mini_cluster.cc:1428] Setting key a65f3e288e4e3f6d6725f6ec3e0216ff
I20260926 16:39:49.777993 25584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:49.778865 25584 log.cc:824] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:49.781328 25584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d: No bootstrap required, opened a new log
Sep 26 16:39:49 dist-test-slave-p39x krb5kdc[25503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:49.785758 25584 raft_consensus.cc:348] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e572c5a9aff84d17baf46b34731d881d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34059 } }
I20260926 16:39:49.786052 25584 raft_consensus.cc:374] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:49.786118 25584 raft_consensus.cc:729] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e572c5a9aff84d17baf46b34731d881d, State: Initialized, Role: FOLLOWER
I20260926 16:39:49.786631 25584 consensus_queue.cc:260] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e572c5a9aff84d17baf46b34731d881d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34059 } }
I20260926 16:39:49.786808 25584 raft_consensus.cc:388] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:39:49.786909 25584 raft_consensus.cc:482] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:39:49.787045 25584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:49.787895 25584 raft_consensus.cc:504] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e572c5a9aff84d17baf46b34731d881d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34059 } }
I20260926 16:39:49.788260 25584 leader_election.cc:302] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e572c5a9aff84d17baf46b34731d881d; no voters:
I20260926 16:39:49.788616 25584 leader_election.cc:288] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:39:49.788712 25589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:39:49.788877 25589 raft_consensus.cc:686] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [term 1 LEADER]: Becoming Leader. State: Replica: e572c5a9aff84d17baf46b34731d881d, State: Running, Role: LEADER
I20260926 16:39:49.789222 25589 consensus_queue.cc:237] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e572c5a9aff84d17baf46b34731d881d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34059 } }
I20260926 16:39:49.789889 25584 sys_catalog.cc:564] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:49.790366 25589 sys_catalog.cc:455] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e572c5a9aff84d17baf46b34731d881d. Latest consensus state: current_term: 1 leader_uuid: "e572c5a9aff84d17baf46b34731d881d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e572c5a9aff84d17baf46b34731d881d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34059 } } }
I20260926 16:39:49.790557 25589 sys_catalog.cc:458] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:49.790757 25590 sys_catalog.cc:455] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e572c5a9aff84d17baf46b34731d881d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e572c5a9aff84d17baf46b34731d881d" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34059 } } }
I20260926 16:39:49.790933 25590 sys_catalog.cc:458] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:49.790977 25596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:39:49.791554 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:49.776889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.11.158.190:34059)
0926 16:39:49.777304 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:49.777324 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:49.777366 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:39:49.778272 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:49.778300 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:49.778741 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:49.779111 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:49.780053 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:49.781510 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:49.782347 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:49.783036 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:49.783203 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:49.785810 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:49.785816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:49.785831 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:49.785860 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:49.787751 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:49.788255 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:49.788261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:49.788273 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:49.788456 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:49.788774 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:49.788779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:49.788781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:49.789296 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:49.790028 (+ 732us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:49.790162 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 16:39:49.790470 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
I20260926 16:39:49.794644 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:39:49.796042 25596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:39:49.801534 25596 catalog_manager.cc:1347] Generated new cluster ID: 88d89133d16149d8b68dcac61e29449e
I20260926 16:39:49.801643 25596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:39:49.810897 25596 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:39:49.811899 25596 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:39:49.822247 25596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e572c5a9aff84d17baf46b34731d881d: Generated new TSK 0
I20260926 16:39:49.822975 25596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:39:49.851202 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25516
2026-09-26T16:39:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1450 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:39:49 dist-test-slave-p39x krb5kdc[25611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:39:49 dist-test-slave-p39x krb5kdc[25611](info): set up 2 sockets
Sep 26 16:39:49 dist-test-slave-p39x krb5kdc[25611](info): commencing operation
krb5kdc: starting...
W20260926 16:39:50.893267 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:39:50 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:39:51.020073 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34959
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:34959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:51.131599 25624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:51.131923 25624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:51.132002 25624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:51.136032 25624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:51.136134 25624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:51.136171 25624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:51.136209 25624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:51.136246 25624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:51.141546 25624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:34959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:51.142622 25624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:51.143666 25624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:51.152242 25632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:51.152233 25629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:51.152282 25624 server_base.cc:1034] running on GCE node
W20260926 16:39:51.152233 25630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:51.153108 25624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:51.154206 25624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:51.155373 25624 hybrid_clock.cc:648] HybridClock initialized: now 1790440791155347 us; error 42 us; skew 500 ppm
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:51.162685 25624 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:39:51.163627 25624 webserver.cc:460] Webserver started at http://127.11.158.190:42811/ using document root <none> and password file <none>
I20260926 16:39:51.164222 25624 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:51.164314 25624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:51.164558 25624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:51.166705 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "9975002eb7a84cd6a4114a47515428cf"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "127390ac2f105bfa53a6d0d958a7552e"
server_key_iv: "0f7f67300210b203fcba976046adcef9"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.167265 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "9975002eb7a84cd6a4114a47515428cf"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "127390ac2f105bfa53a6d0d958a7552e"
server_key_iv: "0f7f67300210b203fcba976046adcef9"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.170933 25624 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260926 16:39:51.173422 25639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.174662 25624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:39:51.174763 25624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "9975002eb7a84cd6a4114a47515428cf"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "127390ac2f105bfa53a6d0d958a7552e"
server_key_iv: "0f7f67300210b203fcba976046adcef9"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.174896 25624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:51.187546 25624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:51.190466 25624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:51.190667 25624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:51.197801 25624 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:34959
I20260926 16:39:51.198150 25691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:34959 every 8 connection(s)
I20260926 16:39:51.198778 25624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:39:51.201853 25692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:51.206811 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25624
I20260926 16:39:51.206933 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:39:51.207230 11898 external_mini_cluster.cc:1428] Setting key 3859ba86053a71d0798cfaf3728d7f04
I20260926 16:39:51.208411 25692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Bootstrap starting.
I20260926 16:39:51.210866 25692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:51.211814 25692 log.cc:824] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:51.214118 25692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: No bootstrap required, opened a new log
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:51.218762 25692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:51.219033 25692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:51.219117 25692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9975002eb7a84cd6a4114a47515428cf, State: Initialized, Role: FOLLOWER
I20260926 16:39:51.219573 25692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:51.219729 25692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:39:51.219823 25692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:39:51.219933 25692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:51.221200 25692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:51.221581 25692 leader_election.cc:302] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9975002eb7a84cd6a4114a47515428cf; no voters:
I20260926 16:39:51.221920 25692 leader_election.cc:288] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:39:51.222036 25697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:39:51.222301 25697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 LEADER]: Becoming Leader. State: Replica: 9975002eb7a84cd6a4114a47515428cf, State: Running, Role: LEADER
I20260926 16:39:51.222626 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.208834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.11.158.190:34959)
0926 16:39:51.209290 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:51.209305 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:51.209340 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:39:51.210012 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:51.210029 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:51.210362 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:51.210650 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:51.211593 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.212716 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.213890 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.214461 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.214602 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.217437 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:51.217442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:51.217457 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:51.217484 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:51.219392 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.219872 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.219878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.219884 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.220051 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.220376 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.220379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.220381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.220897 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:51.221121 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:51.221323 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 16:39:51.221632 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":162,"threads_started":1}
I20260926 16:39:51.222705 25697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:51.223204 25692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:51.223981 25697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9975002eb7a84cd6a4114a47515428cf. Latest consensus state: current_term: 1 leader_uuid: "9975002eb7a84cd6a4114a47515428cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } } }
I20260926 16:39:51.224172 25697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:51.224442 25699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9975002eb7a84cd6a4114a47515428cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } } }
I20260926 16:39:51.224596 25699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:51.225085 25706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:39:51.228256 25706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:39:51.234125 25706 catalog_manager.cc:1347] Generated new cluster ID: 01dee4bf79304369ab9b79ac42d69934
I20260926 16:39:51.234208 25706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:39:51.246235 25706 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:39:51.247223 25706 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:39:51.253368 25706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Generated new TSK 0
I20260926 16:39:51.254181 25706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:39:51.291823 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34959
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:51.405217 25718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:51.405527 25718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:51.405575 25718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:51.409945 25718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:51.410017 25718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:51.410109 25718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:39:51.415581 25718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:51.416764 25718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:51.417886 25718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:51.427983 25726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:51.427924 25724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:51.427939 25723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:51.428117 25718 server_base.cc:1034] running on GCE node
I20260926 16:39:51.428742 25718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:51.429396 25718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:51.430565 25718 hybrid_clock.cc:648] HybridClock initialized: now 1790440791430531 us; error 48 us; skew 500 ppm
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:51.433423 25718 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:39:51.434334 25718 webserver.cc:460] Webserver started at http://127.11.158.129:40861/ using document root <none> and password file <none>
I20260926 16:39:51.434903 25718 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:51.434998 25718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:51.435246 25718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:51.437359 25718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "53abe034f46a4db1bb870546b0d73e01"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "663bce7310dca7fead9edad24ffac14b"
server_key_iv: "47a3530ab73a2a2a17b88f43f615110d"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.437945 25718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "53abe034f46a4db1bb870546b0d73e01"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "663bce7310dca7fead9edad24ffac14b"
server_key_iv: "47a3530ab73a2a2a17b88f43f615110d"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.441715 25718 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:39:51.444520 25733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.445760 25718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:39:51.445904 25718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "53abe034f46a4db1bb870546b0d73e01"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "663bce7310dca7fead9edad24ffac14b"
server_key_iv: "47a3530ab73a2a2a17b88f43f615110d"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.446022 25718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:51.464527 25718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:51.467885 25718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:51.468101 25718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:51.468825 25718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:51.469682 25718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:51.469755 25718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.469827 25718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:51.469870 25718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.480252 25718 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:36609
I20260926 16:39:51.480275 25846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:36609 every 8 connection(s)
I20260926 16:39:51.481228 25718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:39:51.488705 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25718
I20260926 16:39:51.488860 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:39:51.489164 11898 external_mini_cluster.cc:1428] Setting key 4c11e4593af68dd487b4f0f865d0eb61
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:51.496011 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.483099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:43933 (local address 127.11.158.190:34959)
0926 16:39:51.483241 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:51.483244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:51.484143 (+ 899us) server_negotiation.cc:407] Connection header received
0926 16:39:51.485660 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:51.485664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:51.485723 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:51.485834 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:51.487039 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.487586 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.488868 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.489221 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.489342 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.491882 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:51.491885 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:51.491888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:51.491919 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:51.493430 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.494097 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.494100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.494102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.494186 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.494618 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.494621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.494623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.494834 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:51.494925 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:51.495647 (+ 722us) server_negotiation.cc:299] Negotiation successful
0926 16:39:51.495803 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:39:51.497273 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.483470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:43933)
0926 16:39:51.483979 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:51.484025 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:51.485414 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:51.486025 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:51.486032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:51.486399 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:51.486884 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:51.486897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.487718 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.487721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:51.488715 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:51.488725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.489378 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.489380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:51.489544 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.490160 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:51.490176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:51.491714 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:51.493570 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:51.493579 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:51.493599 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:51.493977 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:51.494293 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:51.494299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:51.494304 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:51.494503 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:51.494954 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:51.494965 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:51.495345 (+ 380us) client_negotiation.cc:769] Sending connection context
0926 16:39:51.495585 (+ 240us) client_negotiation.cc:240] Negotiation successful
0926 16:39:51.495985 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":106,"threads_started":1}
I20260926 16:39:51.498764 25847 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34959
I20260926 16:39:51.499049 25847 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:39:51.499619 25847 heartbeater.cc:511] Master 127.11.158.190:34959 requested a full tablet report, sending...
I20260926 16:39:51.501586 25656 ts_manager.cc:194] Registered new tserver with Master: 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609)
I20260926 16:39:51.503080 25656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:43933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:39:51.521859 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34959
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:51.640456 25852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:51.640805 25852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:51.640887 25852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:51.645138 25852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:51.645237 25852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:51.645336 25852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:39:51.651475 25852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:51.652729 25852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:51.653780 25852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:51.663545 25857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:51.663651 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:51.663584 25858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:51.663904 25852 server_base.cc:1034] running on GCE node
I20260926 16:39:51.664292 25852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:51.665000 25852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:51.666167 25852 hybrid_clock.cc:648] HybridClock initialized: now 1790440791666150 us; error 30 us; skew 500 ppm
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:51.669003 25852 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:39:51.669970 25852 webserver.cc:460] Webserver started at http://127.11.158.130:38593/ using document root <none> and password file <none>
I20260926 16:39:51.670554 25852 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:51.670617 25852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:51.670856 25852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:51.673084 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "fdf5553f714a4cc5a01827f7cc0c66b2"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "5b669ecd9ea8d3c7ef6bab0dda7a5de0"
server_key_iv: "95a0cb5b1f54e84730588cb7480985f2"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.673656 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "fdf5553f714a4cc5a01827f7cc0c66b2"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "5b669ecd9ea8d3c7ef6bab0dda7a5de0"
server_key_iv: "95a0cb5b1f54e84730588cb7480985f2"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.677704 25852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:39:51.680325 25867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.681676 25852 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:39:51.681811 25852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "fdf5553f714a4cc5a01827f7cc0c66b2"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "5b669ecd9ea8d3c7ef6bab0dda7a5de0"
server_key_iv: "95a0cb5b1f54e84730588cb7480985f2"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.681927 25852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:51.717202 25852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:51.720851 25852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:51.721099 25852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:51.721753 25852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:51.722636 25852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:51.722708 25852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.722795 25852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:51.722847 25852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.732764 25852 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:39611
I20260926 16:39:51.732795 25980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:39611 every 8 connection(s)
I20260926 16:39:51.733703 25852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:39:51.739835 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25852
I20260926 16:39:51.740017 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:39:51.740298 11898 external_mini_cluster.cc:1428] Setting key 714cb4e7b482f9edc5418127f05077ca
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:39:51.751039 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.735722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:33991 (local address 127.11.158.190:34959)
0926 16:39:51.735915 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:51.735920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:51.737131 (+ 1211us) server_negotiation.cc:407] Connection header received
0926 16:39:51.738516 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:51.738527 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:51.738579 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:51.738660 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:51.739940 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.740772 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.742614 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.742857 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.742972 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.746179 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:51.746189 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:51.746191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:51.746227 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:51.748412 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.749279 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.749282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.749284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.749379 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.749736 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.749740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.749742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.749996 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:51.750112 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:51.750736 (+ 624us) server_negotiation.cc:299] Negotiation successful
0926 16:39:51.750872 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 16:39:51.751919 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.736263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:33991)
0926 16:39:51.736936 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:51.736998 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:51.738286 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:51.738837 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:51.738843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:51.739241 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:51.739764 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:51.739780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.740927 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.740931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:51.742465 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:51.742478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.743046 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.743050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:51.743246 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.744000 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:51.744035 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:51.745960 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:51.748628 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:51.748637 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:51.748728 (+ 91us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:51.749062 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:51.749481 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:51.749483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:51.749485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:51.749633 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:51.750112 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:51.750121 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:51.750473 (+ 352us) client_negotiation.cc:769] Sending connection context
0926 16:39:51.750682 (+ 209us) client_negotiation.cc:240] Negotiation successful
0926 16:39:51.751055 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":160,"threads_started":1}
I20260926 16:39:51.753306 25981 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34959
I20260926 16:39:51.753611 25981 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:39:51.754181 25981 heartbeater.cc:511] Master 127.11.158.190:34959 requested a full tablet report, sending...
I20260926 16:39:51.755362 25656 ts_manager.cc:194] Registered new tserver with Master: fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611)
I20260926 16:39:51.756275 25656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:33991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:39:51.773170 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34959
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:51.891274 25986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:51.891614 25986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:51.891701 25986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:51.895864 25986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:51.895987 25986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:51.896096 25986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:39:51.901834 25986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:51.903055 25986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:51.904182 25986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:51.914866 25991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:51.914876 25994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:51.914966 25986 server_base.cc:1034] running on GCE node
W20260926 16:39:51.914888 25992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:51.915673 25986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:51.916396 25986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:51.917600 25986 hybrid_clock.cc:648] HybridClock initialized: now 1790440791917580 us; error 32 us; skew 500 ppm
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:51.920922 25986 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:39:51.921905 25986 webserver.cc:460] Webserver started at http://127.11.158.131:34869/ using document root <none> and password file <none>
I20260926 16:39:51.922508 25986 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:51.922572 25986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:51.922813 25986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:39:51.925241 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "0a4c7dd604a64e508d66349cdff026df"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "d6d9712dc950fbb1c83b7940caf4b45c"
server_key_iv: "ed546a196979f83a622d3bd68b1555d7"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.925860 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "0a4c7dd604a64e508d66349cdff026df"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "d6d9712dc950fbb1c83b7940caf4b45c"
server_key_iv: "ed546a196979f83a622d3bd68b1555d7"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.929903 25986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:39:51.932476 26001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.933952 25986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:51.934084 25986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "0a4c7dd604a64e508d66349cdff026df"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "d6d9712dc950fbb1c83b7940caf4b45c"
server_key_iv: "ed546a196979f83a622d3bd68b1555d7"
server_key_version: "encryptionkey@0"
I20260926 16:39:51.934228 25986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:51.946164 25986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:51.949499 25986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:51.949755 25986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:51.950421 25986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:51.951371 25986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:51.951443 25986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.951522 25986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:51.951576 25986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:51.960481 25986 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:42829
I20260926 16:39:51.960527 26114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:42829 every 8 connection(s)
I20260926 16:39:51.961395 25986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:39:51.970445 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 25986
I20260926 16:39:51.970602 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:39:51.970934 11898 external_mini_cluster.cc:1428] Setting key fcf35b07e37ad19be211536ae0de9e76
Sep 26 16:39:51 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440791, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:51.975888 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.963324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:60459 (local address 127.11.158.190:34959)
0926 16:39:51.963488 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:51.963492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:51.964419 (+ 927us) server_negotiation.cc:407] Connection header received
0926 16:39:51.965847 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:51.965851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:51.965914 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:51.966036 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:51.967266 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.967823 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.969039 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.969273 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.969417 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.972421 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:51.972431 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:51.972434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:51.972459 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:51.974017 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.974496 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.974499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.974501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.974581 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.974886 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.974889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.974890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.975068 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:51.975148 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:51.975660 (+ 512us) server_negotiation.cc:299] Negotiation successful
0926 16:39:51.975766 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:39:51.976850 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.963693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:60459)
0926 16:39:51.964234 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:51.964280 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:51.965614 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:51.966212 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:51.966220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:51.966599 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:51.967098 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:51.967111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.967965 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.967969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:51.968895 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:51.968907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.969461 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.969466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:51.969646 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.970253 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:51.970277 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:51.972221 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:51.974123 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:51.974128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:51.974141 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:51.974394 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:51.974674 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:51.974676 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:51.974678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:51.974804 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:51.975167 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:51.975172 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:51.975409 (+ 237us) client_negotiation.cc:769] Sending connection context
0926 16:39:51.975617 (+ 208us) client_negotiation.cc:240] Negotiation successful
0926 16:39:51.975885 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":125,"threads_started":1}
I20260926 16:39:51.978063 26115 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34959
I20260926 16:39:51.978327 26115 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:51.978847 26115 heartbeater.cc:511] Master 127.11.158.190:34959 requested a full tablet report, sending...
I20260926 16:39:51.979882 25656 ts_manager.cc:194] Registered new tserver with Master: 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829)
I20260926 16:39:51.980477 25656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:60459
I20260926 16:39:51.984855 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:39:51.994400 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:51.986865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.11.158.190:34959)
0926 16:39:51.987012 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:51.987017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:51.987205 (+ 188us) server_negotiation.cc:407] Connection header received
0926 16:39:51.987384 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:51.987386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:51.987427 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:51.987513 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:51.988234 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.988819 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.989682 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:51.989903 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:51.990020 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:51.991434 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:51.991440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:51.991447 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:51.991471 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:51.992721 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.993145 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.993151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.993152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.993233 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:51.993515 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:51.993519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:51.993520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:51.993735 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:51.993873 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:51.994122 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 16:39:51.994232 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:39:51.997241 25656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:39:52.000160 25656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:39:52.015666 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.010336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.11.158.129:36609)
0926 16:39:52.010622 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.010630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.010655 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:39:52.010775 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.010781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.010986 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.011140 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:52.011471 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.012756 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.014358 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.014898 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.015038 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.015184 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.015414 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.015515 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":90,"threads_started":1}
I20260926 16:39:52.015657 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.009482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36609 (local address 127.0.0.1:55990)
0926 16:39:52.010561 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.010575 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.010655 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.011146 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.011152 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.011175 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:52.011332 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.011340 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.012914 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.012917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.014229 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.014235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.015023 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.015027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.015175 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.015226 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.015379 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.015506 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":96,"thread_start_us":85,"threads_started":1}
I20260926 16:39:52.015852 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.009901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.11.158.131:42829)
0926 16:39:52.010737 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.010747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.010765 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:39:52.010825 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.010829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.010985 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.011140 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:52.011676 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.012483 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.014343 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.015119 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.015225 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.015332 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.015621 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.015717 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":173,"threads_started":1}
I20260926 16:39:52.015843 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.009196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:42829 (local address 127.0.0.1:36278)
0926 16:39:52.010081 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.010139 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.010278 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.011356 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.011363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.011384 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:52.011547 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.011554 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.012606 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.012611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.014212 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.014223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.015402 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.015405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.015476 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.015495 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.015589 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.015662 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":342,"spinlock_wait_cycles":3200,"thread_start_us":86,"threads_started":1}
I20260926 16:39:52.016331 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.009196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39611 (local address 127.0.0.1:45680)
0926 16:39:52.010081 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.010139 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.010278 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.011382 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.011384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.011396 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:52.011526 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.011534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.012914 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.012917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.014011 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.014020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.015103 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.015107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.016040 (+ 933us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.016063 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.016135 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.016204 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":866,"spinlock_wait_cycles":7808,"thread_start_us":98,"threads_started":1}
I20260926 16:39:52.016479 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.009360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.11.158.130:39611)
0926 16:39:52.010905 (+ 1545us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.010914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.010932 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:39:52.010990 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.010994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.011179 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.011376 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:52.011648 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.012779 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.014136 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.014953 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.015208 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.015332 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.016170 (+ 838us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.016278 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1454,"thread_start_us":106,"threads_started":1}
I20260926 16:39:52.018748 25781 tablet_service.cc:1467] Processing CreateTablet for tablet 1818619d13984536a2d9dd7e5c2500dc (DEFAULT_TABLE table=test-table [id=2c9c23dd700d4506ad28898544536978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:52.018958 26049 tablet_service.cc:1467] Processing CreateTablet for tablet 1818619d13984536a2d9dd7e5c2500dc (DEFAULT_TABLE table=test-table [id=2c9c23dd700d4506ad28898544536978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:52.018962 25915 tablet_service.cc:1467] Processing CreateTablet for tablet 1818619d13984536a2d9dd7e5c2500dc (DEFAULT_TABLE table=test-table [id=2c9c23dd700d4506ad28898544536978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:52.019809 25781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1818619d13984536a2d9dd7e5c2500dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:52.019946 25915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1818619d13984536a2d9dd7e5c2500dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:52.019946 26049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1818619d13984536a2d9dd7e5c2500dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:52.026710 26132 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Bootstrap starting.
I20260926 16:39:52.028080 26134 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Bootstrap starting.
I20260926 16:39:52.028813 26133 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Bootstrap starting.
I20260926 16:39:52.029166 26132 tablet_bootstrap.cc:654] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:52.030224 26132 log.cc:824] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:52.031002 26134 tablet_bootstrap.cc:654] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:52.031569 26133 tablet_bootstrap.cc:654] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:52.032002 26134 log.cc:824] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:52.032470 26133 log.cc:824] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:52.032859 26132 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: No bootstrap required, opened a new log
I20260926 16:39:52.033098 26132 ts_tablet_manager.cc:1397] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 16:39:52.033969 26134 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: No bootstrap required, opened a new log
I20260926 16:39:52.034174 26134 ts_tablet_manager.cc:1397] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260926 16:39:52.036727 26132 raft_consensus.cc:348] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.037020 26132 raft_consensus.cc:374] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:52.037089 26132 raft_consensus.cc:729] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53abe034f46a4db1bb870546b0d73e01, State: Initialized, Role: FOLLOWER
I20260926 16:39:52.037433 26134 raft_consensus.cc:348] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.037563 26132 consensus_queue.cc:260] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.037655 26134 raft_consensus.cc:374] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:52.037709 26134 raft_consensus.cc:729] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4c7dd604a64e508d66349cdff026df, State: Initialized, Role: FOLLOWER
I20260926 16:39:52.038198 26134 consensus_queue.cc:260] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.038335 25847 heartbeater.cc:503] Master 127.11.158.190:34959 was elected leader, sending a full tablet report...
I20260926 16:39:52.038715 26132 ts_tablet_manager.cc:1428] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:39:52.039228 26133 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: No bootstrap required, opened a new log
I20260926 16:39:52.039234 26134 ts_tablet_manager.cc:1428] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 16:39:52.039448 26133 ts_tablet_manager.cc:1397] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260926 16:39:52.039552 26115 heartbeater.cc:503] Master 127.11.158.190:34959 was elected leader, sending a full tablet report...
I20260926 16:39:52.043346 26133 raft_consensus.cc:348] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.043641 26133 raft_consensus.cc:374] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:52.043691 26133 raft_consensus.cc:729] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf5553f714a4cc5a01827f7cc0c66b2, State: Initialized, Role: FOLLOWER
I20260926 16:39:52.044032 26133 consensus_queue.cc:260] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.044690 25981 heartbeater.cc:503] Master 127.11.158.190:34959 was elected leader, sending a full tablet report...
I20260926 16:39:52.044984 26133 ts_tablet_manager.cc:1428] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:39:52.051517 26139 raft_consensus.cc:482] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:39:52.051818 26139 raft_consensus.cc:504] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.053108 26139 leader_election.cc:288] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611), 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609)
I20260926 16:39:52.058285 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.053321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39611 (local address 127.11.158.131:42325)
0926 16:39:52.053543 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.053559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.053657 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.053882 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.053886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.053924 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:52.054064 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.054069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.055093 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.055096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.056647 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.056660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.057786 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.057790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.057901 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.057926 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.058017 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.058109 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":86,"spinlock_wait_cycles":27776}
I20260926 16:39:52.058327 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.053447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:42325 (local address 127.11.158.130:39611)
0926 16:39:52.053674 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.053678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.053692 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:39:52.053737 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.053739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.053779 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.053846 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:52.054171 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.054963 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.056791 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.057421 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.057580 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.057633 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.058098 (+ 465us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.058187 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260926 16:39:52.058926 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.053342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36609 (local address 127.11.158.131:56615)
0926 16:39:52.053791 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.053801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.053931 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.054199 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.054202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.054216 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:52.054298 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.054302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.055241 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.055244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.056647 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.056660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.058518 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.058521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.058611 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.058630 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.058715 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.058785 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":45,"spinlock_wait_cycles":8576,"thread_start_us":119,"threads_started":1}
I20260926 16:39:52.058962 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.053468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:56615 (local address 127.11.158.129:36609)
0926 16:39:52.053602 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.053606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.053917 (+ 311us) server_negotiation.cc:407] Connection header received
0926 16:39:52.054038 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.054041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.054079 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.054173 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:52.054414 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.055126 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.056791 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.057421 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.057607 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.057653 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.058744 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.058826 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:39:52.059222 25935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1818619d13984536a2d9dd7e5c2500dc" candidate_uuid: "0a4c7dd604a64e508d66349cdff026df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" is_pre_election: true
I20260926 16:39:52.059530 25801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1818619d13984536a2d9dd7e5c2500dc" candidate_uuid: "0a4c7dd604a64e508d66349cdff026df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53abe034f46a4db1bb870546b0d73e01" is_pre_election: true
I20260926 16:39:52.059600 25935 raft_consensus.cc:2454] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4c7dd604a64e508d66349cdff026df in term 0.
I20260926 16:39:52.059810 25801 raft_consensus.cc:2454] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4c7dd604a64e508d66349cdff026df in term 0.
I20260926 16:39:52.060262 26002 leader_election.cc:302] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a4c7dd604a64e508d66349cdff026df, 53abe034f46a4db1bb870546b0d73e01, fdf5553f714a4cc5a01827f7cc0c66b2; no voters:
I20260926 16:39:52.060636 26139 raft_consensus.cc:2781] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:39:52.060734 26139 raft_consensus.cc:482] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:39:52.060796 26139 raft_consensus.cc:3037] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:52.061919 26139 raft_consensus.cc:504] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.062315 26139 leader_election.cc:288] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [CANDIDATE]: Term 1 election: Requested vote from peers fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611), 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609)
I20260926 16:39:52.062738 25801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1818619d13984536a2d9dd7e5c2500dc" candidate_uuid: "0a4c7dd604a64e508d66349cdff026df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53abe034f46a4db1bb870546b0d73e01"
I20260926 16:39:52.062752 25935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1818619d13984536a2d9dd7e5c2500dc" candidate_uuid: "0a4c7dd604a64e508d66349cdff026df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2"
I20260926 16:39:52.062891 25801 raft_consensus.cc:3037] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:52.062898 25935 raft_consensus.cc:3037] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:52.064010 25935 raft_consensus.cc:2454] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4c7dd604a64e508d66349cdff026df in term 1.
I20260926 16:39:52.064268 25801 raft_consensus.cc:2454] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4c7dd604a64e508d66349cdff026df in term 1.
I20260926 16:39:52.064405 26002 leader_election.cc:302] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4c7dd604a64e508d66349cdff026df, fdf5553f714a4cc5a01827f7cc0c66b2; no voters:
I20260926 16:39:52.064635 26139 raft_consensus.cc:2781] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:39:52.064888 26139 raft_consensus.cc:686] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 1 LEADER]: Becoming Leader. State: Replica: 0a4c7dd604a64e508d66349cdff026df, State: Running, Role: LEADER
I20260926 16:39:52.065258 26139 consensus_queue.cc:237] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.068596 25656 catalog_manager.cc:5495] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a4c7dd604a64e508d66349cdff026df (127.11.158.131). New cstate: current_term: 1 leader_uuid: "0a4c7dd604a64e508d66349cdff026df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260926 16:39:52.096007 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.092500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.11.158.131:42829)
0926 16:39:52.092648 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.092652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.092664 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:39:52.092738 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.092740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.092792 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.092909 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:39:52.093170 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.093679 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.094537 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.094760 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.094873 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.095196 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:39:52.095615 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:39:52.095753 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.095797 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.095854 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 16:39:52.102195 25935 raft_consensus.cc:1261] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a4c7dd604a64e508d66349cdff026df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:52.102195 25801 raft_consensus.cc:1261] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Refusing update from remote peer 0a4c7dd604a64e508d66349cdff026df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:52.103133 26142 consensus_queue.cc:1035] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:39:52.103379 26139 consensus_queue.cc:1035] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [LEADER]: Connected to new peer: Peer: permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:39:52.111483 26147 mvcc.cc:205] Tried to move back new op lower bound from 7333645484442243072 to 7333645484302508032. Current Snapshot: MvccSnapshot[applied={T|T < 7333645484442243072}]
I20260926 16:39:52.120007 25655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:39:52.120162 25655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:39:52.122658 25655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Sending DeleteTablet for 3 replicas of tablet 1818619d13984536a2d9dd7e5c2500dc
I20260926 16:39:52.123539 25915 tablet_service.cc:1514] Processing DeleteTablet for tablet 1818619d13984536a2d9dd7e5c2500dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:52 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:45680
I20260926 16:39:52.123591 25781 tablet_service.cc:1514] Processing DeleteTablet for tablet 1818619d13984536a2d9dd7e5c2500dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:52 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:55990
I20260926 16:39:52.123742 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25718
I20260926 16:39:52.123826 26049 tablet_service.cc:1514] Processing DeleteTablet for tablet 1818619d13984536a2d9dd7e5c2500dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:52 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:36278
I20260926 16:39:52.125115 26154 tablet_replica.cc:330] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: stopping tablet replica
I20260926 16:39:52.125519 26154 raft_consensus.cc:2229] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:39:52.125921 26154 raft_consensus.cc:2258] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:39:52.128959 26154 ts_tablet_manager.cc:1905] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:52.132030 26154 ts_tablet_manager.cc:1918] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:52.132169 26154 log.cc:1196] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/1818619d13984536a2d9dd7e5c2500dc
I20260926 16:39:52.132532 26154 ts_tablet_manager.cc:1939] T 1818619d13984536a2d9dd7e5c2500dc P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting consensus metadata
I20260926 16:39:52.133855 25640 catalog_manager.cc:4841] TS fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611): tablet 1818619d13984536a2d9dd7e5c2500dc (table test-table [id=2c9c23dd700d4506ad28898544536978]) successfully deleted
I20260926 16:39:52.134246 26155 tablet_replica.cc:330] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: stopping tablet replica
I20260926 16:39:52.134567 26155 raft_consensus.cc:2229] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:39:52.134953 26155 raft_consensus.cc:2258] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 16:39:52.135082 25642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:36609 (error 108)
I20260926 16:39:52.135293 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25852
I20260926 16:39:52.137010 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.136516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36609 (local address 127.0.0.1:56000)
0926 16:39:52.136850 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.136923 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:36609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
W20260926 16:39:52.138134 25642 catalog_manager.cc:4584] TS 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1818619d13984536a2d9dd7e5c2500dc: Network error: Client connection negotiation failed: client connection to 127.11.158.129:36609: connect: Connection refused (error 111)
I20260926 16:39:52.138329 26155 ts_tablet_manager.cc:1905] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:52.140419 26155 ts_tablet_manager.cc:1918] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:52.140547 26155 log.cc:1196] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/1818619d13984536a2d9dd7e5c2500dc
I20260926 16:39:52.140861 26155 ts_tablet_manager.cc:1939] T 1818619d13984536a2d9dd7e5c2500dc P 0a4c7dd604a64e508d66349cdff026df: Deleting consensus metadata
I20260926 16:39:52.141911 25641 catalog_manager.cc:4841] TS 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829): tablet 1818619d13984536a2d9dd7e5c2500dc (table test-table [id=2c9c23dd700d4506ad28898544536978]) successfully deleted
I20260926 16:39:52.143903 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25986
I20260926 16:39:52.148834 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 25624
I20260926 16:39:52.155119 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34959
--webserver_interface=127.11.158.190
--webserver_port=42811
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:34959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:52.266573 26157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:52.266901 26157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:52.266978 26157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:52.271298 26157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:52.271389 26157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:52.271427 26157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:52.271467 26157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:52.271507 26157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:52.277212 26157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:34959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:52.278404 26157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:52.279508 26157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:52.292985 26166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:52.293000 26164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:52.292965 26163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:52.293296 26157 server_base.cc:1034] running on GCE node
I20260926 16:39:52.293979 26157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:52.295183 26157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:52.296399 26157 hybrid_clock.cc:648] HybridClock initialized: now 1790440792296380 us; error 44 us; skew 500 ppm
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:52.299196 26157 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:39:52.300096 26157 webserver.cc:460] Webserver started at http://127.11.158.190:42811/ using document root <none> and password file <none>
I20260926 16:39:52.300746 26157 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:52.300860 26157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:52.305037 26157 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:39:52.307272 26173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.308705 26157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:39:52.308859 26157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "9975002eb7a84cd6a4114a47515428cf"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "127390ac2f105bfa53a6d0d958a7552e"
server_key_iv: "0f7f67300210b203fcba976046adcef9"
server_key_version: "encryptionkey@0"
I20260926 16:39:52.309500 26157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:52.330607 26157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:52.334034 26157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:52.334247 26157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:52.341264 26157 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:34959
I20260926 16:39:52.341563 26225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:34959 every 8 connection(s)
I20260926 16:39:52.342420 26157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:39:52.346906 26226 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:39:52.348654 26226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Bootstrap starting.
I20260926 16:39:52.351922 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 26157
I20260926 16:39:52.352524 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:36609
--local_ip_for_outbound_sockets=127.11.158.129
--tserver_master_addrs=127.11.158.190:34959
--webserver_port=40861
--webserver_interface=127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:39:52.370182 26226 log.cc:824] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:52.381431 26226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:39:52.381865 26226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Bootstrap complete.
I20260926 16:39:52.387343 26226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:52.387876 26226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9975002eb7a84cd6a4114a47515428cf, State: Initialized, Role: FOLLOWER
I20260926 16:39:52.388574 26226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:52.388758 26226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:39:52.388826 26226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:39:52.388924 26226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:39:52.391382 26226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:52.391754 26226 leader_election.cc:302] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9975002eb7a84cd6a4114a47515428cf; no voters:
I20260926 16:39:52.392124 26226 leader_election.cc:288] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:39:52.392213 26230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:39:52.392542 26230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 LEADER]: Becoming Leader. State: Replica: 9975002eb7a84cd6a4114a47515428cf, State: Running, Role: LEADER
I20260926 16:39:52.392952 26230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:52.393491 26226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:52.394554 26231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9975002eb7a84cd6a4114a47515428cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } } }
I20260926 16:39:52.394687 26231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:52.394536 26232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9975002eb7a84cd6a4114a47515428cf. Latest consensus state: current_term: 2 leader_uuid: "9975002eb7a84cd6a4114a47515428cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } } }
I20260926 16:39:52.394800 26232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:52.395082 26239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:39:52.399986 26239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:39:52.400839 26239 catalog_manager.cc:1259] Loaded cluster ID: 01dee4bf79304369ab9b79ac42d69934
I20260926 16:39:52.400916 26239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:39:52.402005 26239 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 16:39:52.404855 26239 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 16:39:52.409574 26239 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:39:52.410307 26239 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Loaded TSK: 0
I20260926 16:39:52.411267 26239 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:39:52.488132 26228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:52.488484 26228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:52.488574 26228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:52.493104 26228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:52.493186 26228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:52.493287 26228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:39:52.499068 26228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:36609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=40861
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:52.500089 26228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:52.501140 26228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:52.511585 26253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:52.511582 26254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:52.511618 26256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:52.512250 26228 server_base.cc:1034] running on GCE node
I20260926 16:39:52.512701 26228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:52.513423 26228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:52.514605 26228 hybrid_clock.cc:648] HybridClock initialized: now 1790440792514576 us; error 50 us; skew 500 ppm
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:52.517498 26228 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:39:52.518527 26228 webserver.cc:460] Webserver started at http://127.11.158.129:40861/ using document root <none> and password file <none>
I20260926 16:39:52.519106 26228 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:52.519227 26228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:52.523334 26228 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:39:52.525738 26263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.527143 26228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:39:52.527273 26228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "53abe034f46a4db1bb870546b0d73e01"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "663bce7310dca7fead9edad24ffac14b"
server_key_iv: "47a3530ab73a2a2a17b88f43f615110d"
server_key_version: "encryptionkey@0"
I20260926 16:39:52.527850 26228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:52.537354 26228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:52.540804 26228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:52.541033 26228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:52.541690 26228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:52.542807 26270 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:39:52.547389 26228 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:39:52.547464 26228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260926 16:39:52.547519 26228 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:39:52.549341 26228 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:39:52.549402 26228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260926 16:39:52.549693 26270 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Bootstrap starting.
I20260926 16:39:52.560055 26228 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:36609
I20260926 16:39:52.560107 26377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:36609 every 8 connection(s)
I20260926 16:39:52.561355 26228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:39:52.570200 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 26228
I20260926 16:39:52.570267 26270 log.cc:824] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:52.570712 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:39611
--local_ip_for_outbound_sockets=127.11.158.130
--tserver_master_addrs=127.11.158.190:34959
--webserver_port=38593
--webserver_interface=127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:52.578987 26270 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:39:52.579152 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.563565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:46065 (local address 127.11.158.190:34959)
0926 16:39:52.563938 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.563958 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.564266 (+ 308us) server_negotiation.cc:407] Connection header received
0926 16:39:52.565495 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.565506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.565790 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.566129 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:52.567260 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.568222 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.569410 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.569908 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.570084 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.573047 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:52.573054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:52.573071 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:52.573107 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:52.575335 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.576044 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.576050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.576056 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.576212 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.576696 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.576700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.576702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.577150 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:52.577286 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.577991 (+ 705us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.578281 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":121,"threads_started":1}
I20260926 16:39:52.579433 26270 tablet_bootstrap.cc:492] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Bootstrap complete.
I20260926 16:39:52.579728 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.563597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:46065)
0926 16:39:52.564107 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.564148 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.565004 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.566179 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.566187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.566623 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:52.567074 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.567088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.568427 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.568430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.569275 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.569285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.570059 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.570063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.570255 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.570971 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:52.570994 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:52.572834 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:52.575473 (+ 2639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.575483 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.575499 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.575891 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.576324 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.576329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.576369 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.576567 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.577281 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:52.577289 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:52.577616 (+ 327us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.577834 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.578226 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":138,"threads_started":1}
I20260926 16:39:52.580767 26270 ts_tablet_manager.cc:1397] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Time spent bootstrapping tablet: real 0.031s user 0.019s sys 0.012s
I20260926 16:39:52.582551 26378 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34959
I20260926 16:39:52.582778 26378 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:52.583298 26378 heartbeater.cc:511] Master 127.11.158.190:34959 requested a full tablet report, sending...
I20260926 16:39:52.583415 26270 raft_consensus.cc:348] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.583935 26270 raft_consensus.cc:729] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53abe034f46a4db1bb870546b0d73e01, State: Initialized, Role: FOLLOWER
I20260926 16:39:52.584326 26270 consensus_queue.cc:260] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.585423 26270 ts_tablet_manager.cc:1428] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260926 16:39:52.587461 26190 ts_manager.cc:194] Registered new tserver with Master: 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609)
I20260926 16:39:52.591421 26190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:46065
I20260926 16:39:52.594092 26378 heartbeater.cc:503] Master 127.11.158.190:34959 was elected leader, sending a full tablet report...
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:39:52.601459 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.590787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36609 (local address 127.0.0.1:56004)
0926 16:39:52.591160 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.591202 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.591336 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.591986 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.591992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.592493 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:52.592602 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.592608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.593988 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.593991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.594812 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.594822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.595506 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.595511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.595648 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.596237 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:52.596254 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:52.597730 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:52.599632 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.599643 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.599649 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.599885 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.600202 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.600206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.600208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.600296 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.600807 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:52.600813 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:52.600944 (+ 131us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.601079 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.601276 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":107,"threads_started":1}
I20260926 16:39:52.601490 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.590823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.11.158.129:36609)
0926 16:39:52.591596 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.591612 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.591638 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:39:52.591691 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.591695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.591838 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.591984 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:52.592724 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.593867 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.594989 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.595363 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.595483 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.597876 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:52.597885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:52.597907 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:52.597929 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:52.599513 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.600013 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.600023 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.600025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.600100 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.600453 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.600456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.600458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.600687 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:52.600832 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.601172 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.601330 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":90,"threads_started":1}
I20260926 16:39:52.602450 26312 tablet_service.cc:1514] Processing DeleteTablet for tablet 1818619d13984536a2d9dd7e5c2500dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:52 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:56004
I20260926 16:39:52.603461 26387 tablet_replica.cc:330] stopping tablet replica
I20260926 16:39:52.603785 26387 raft_consensus.cc:2229] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:39:52.604046 26387 raft_consensus.cc:2258] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:39:52.605077 26387 ts_tablet_manager.cc:1905] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:52.608670 26387 ts_tablet_manager.cc:1918] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:52.608791 26387 log.cc:1196] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/1818619d13984536a2d9dd7e5c2500dc
I20260926 16:39:52.609092 26387 ts_tablet_manager.cc:1939] T 1818619d13984536a2d9dd7e5c2500dc P 53abe034f46a4db1bb870546b0d73e01: Deleting consensus metadata
I20260926 16:39:52.610728 26176 catalog_manager.cc:4841] TS 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609): tablet 1818619d13984536a2d9dd7e5c2500dc (table test-table [id=2c9c23dd700d4506ad28898544536978]) successfully deleted
W20260926 16:39:52.690164 26383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:52.690402 26383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:52.690445 26383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:52.694746 26383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:52.694810 26383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:52.694897 26383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:39:52.700394 26383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:39611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=38593
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:52.701496 26383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:52.702529 26383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:52.712414 26393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:52.712433 26392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:52.712536 26383 server_base.cc:1034] running on GCE node
W20260926 16:39:52.712430 26395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:52.713128 26383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:52.713799 26383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:52.714957 26383 hybrid_clock.cc:648] HybridClock initialized: now 1790440792714916 us; error 55 us; skew 500 ppm
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:52.717782 26383 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:39:52.718730 26383 webserver.cc:460] Webserver started at http://127.11.158.130:38593/ using document root <none> and password file <none>
I20260926 16:39:52.719296 26383 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:52.719368 26383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:52.723212 26383 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:39:52.725629 26402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.726871 26383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:52.727020 26383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "fdf5553f714a4cc5a01827f7cc0c66b2"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "5b669ecd9ea8d3c7ef6bab0dda7a5de0"
server_key_iv: "95a0cb5b1f54e84730588cb7480985f2"
server_key_version: "encryptionkey@0"
I20260926 16:39:52.727574 26383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:52.738430 26383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:52.741720 26383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:52.741923 26383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:52.742501 26383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:52.743316 26383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:52.743394 26383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.743465 26383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:52.743516 26383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.751941 26383 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:39611
I20260926 16:39:52.752031 26515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:39611 every 8 connection(s)
I20260926 16:39:52.753041 26383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:39:52.758100 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 26383
I20260926 16:39:52.758515 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:42829
--local_ip_for_outbound_sockets=127.11.158.131
--tserver_master_addrs=127.11.158.190:34959
--webserver_port=34869
--webserver_interface=127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:52.767320 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.754864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:56653 (local address 127.11.158.190:34959)
0926 16:39:52.755029 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.755033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.755852 (+ 819us) server_negotiation.cc:407] Connection header received
0926 16:39:52.757377 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.757381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.757453 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.757544 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:52.758862 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.759403 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.760645 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.760900 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.761020 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.763509 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:52.763512 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:52.763514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:52.763542 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:52.765086 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.765635 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.765639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.765641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.765766 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.766136 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.766141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.766143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.766340 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:52.766462 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.766993 (+ 531us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.767145 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 16:39:52.768172 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.755246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:56653)
0926 16:39:52.755700 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.755742 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.757148 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.757722 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.757730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.758088 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:52.758639 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.758658 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.759536 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.759539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.760516 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.760525 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.761063 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.761068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.761245 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.761863 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:52.761880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:52.763313 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:52.765211 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.765217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.765234 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.765521 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.765874 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.765877 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.765879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.766036 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.766458 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:52.766466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:52.766736 (+ 270us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.766941 (+ 205us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.767307 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":105,"threads_started":1}
I20260926 16:39:52.769430 26516 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34959
I20260926 16:39:52.769708 26516 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:52.770231 26516 heartbeater.cc:511] Master 127.11.158.190:34959 requested a full tablet report, sending...
I20260926 16:39:52.771461 26190 ts_manager.cc:194] Registered new tserver with Master: fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611)
I20260926 16:39:52.772107 26190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:56653
W20260926 16:39:52.871062 26519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:52.871335 26519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:52.871380 26519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:52.875586 26519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:52.875648 26519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:52.875708 26519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:39:52.881108 26519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:42829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=34869
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:52.882174 26519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:52.883198 26519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:52.892691 26525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:52.892704 26527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:52.892830 26524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:52.892966 26519 server_base.cc:1034] running on GCE node
I20260926 16:39:52.893347 26519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:52.894116 26519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:52.895298 26519 hybrid_clock.cc:648] HybridClock initialized: now 1790440792895279 us; error 42 us; skew 500 ppm
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:52.898039 26519 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:39:52.898908 26519 webserver.cc:460] Webserver started at http://127.11.158.131:34869/ using document root <none> and password file <none>
I20260926 16:39:52.899456 26519 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:52.899540 26519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:52.903323 26519 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260926 16:39:52.905529 26534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.906761 26519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:39:52.906878 26519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "0a4c7dd604a64e508d66349cdff026df"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "d6d9712dc950fbb1c83b7940caf4b45c"
server_key_iv: "ed546a196979f83a622d3bd68b1555d7"
server_key_version: "encryptionkey@0"
I20260926 16:39:52.907546 26519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:52.925207 26519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:52.928686 26519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:52.928896 26519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:52.929517 26519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:52.930346 26519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:52.930406 26519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.930493 26519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:52.930550 26519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:52.939535 26519 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:42829
I20260926 16:39:52.939563 26647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:42829 every 8 connection(s)
I20260926 16:39:52.940541 26519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:39:52.945099 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 26519
Sep 26 16:39:52 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440792, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:39:52.953927 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.945778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.11.158.190:34959)
0926 16:39:52.946045 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.946048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.946166 (+ 118us) server_negotiation.cc:407] Connection header received
0926 16:39:52.946279 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.946282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.946325 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.946416 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:52.947164 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.947647 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.948749 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.948978 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.949129 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.950540 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:52.950546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:52.950552 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:52.950584 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:52.952143 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.952648 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.952652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.952655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.952750 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.953055 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.953058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.953059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.953222 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:52.953368 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.953611 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.953733 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":102,"threads_started":1}
I20260926 16:39:52.955466 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.942352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:53935 (local address 127.11.158.190:34959)
0926 16:39:52.942531 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.942536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.943329 (+ 793us) server_negotiation.cc:407] Connection header received
0926 16:39:52.944706 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.944709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.944754 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.944852 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:52.946195 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.946831 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.948197 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.948481 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.948562 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.951675 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:52.951677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:52.951680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:52.951713 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:52.953339 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.953921 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.953924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.953925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.954041 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:52.954377 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:52.954380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:52.954384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:52.954583 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:52.954677 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.955276 (+ 599us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.955361 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 16:39:52.956454 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.942748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:53935)
0926 16:39:52.943183 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.943223 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.944518 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.944996 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.945003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.945409 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:52.946042 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.946061 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.946957 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.946961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.948074 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.948086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.948625 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.948630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.948822 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.949426 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:52.949465 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:52.951493 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:52.953462 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.953470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.953486 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.953826 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.954128 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:52.954131 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:52.954133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:52.954276 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:52.954691 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:52.954696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:52.955023 (+ 327us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.955241 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.955570 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":104,"threads_started":1}
I20260926 16:39:52.957422 26648 heartbeater.cc:347] Connected to a master server at 127.11.158.190:34959
I20260926 16:39:52.957662 26648 heartbeater.cc:464] Registering TS with master...
I20260926 16:39:52.958137 26648 heartbeater.cc:511] Master 127.11.158.190:34959 requested a full tablet report, sending...
I20260926 16:39:52.959098 26190 ts_manager.cc:194] Registered new tserver with Master: 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829)
I20260926 16:39:52.959643 26190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:53935
I20260926 16:39:52.967547 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:39:52.971536 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.968170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.11.158.190:34959)
0926 16:39:52.968286 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.968289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.968501 (+ 212us) server_negotiation.cc:407] Connection header received
0926 16:39:52.968621 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.968623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.968719 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.968802 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:39:52.968981 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.969464 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.970305 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.970516 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.970658 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.970890 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:39:52.971257 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:39:52.971342 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.971396 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.971439 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:39:52.972175 26190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:39:52.975360 26190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:39:52.983593 26312 tablet_service.cc:1467] Processing CreateTablet for tablet 7a56b77c77a64d98be7df535649a6ddd (DEFAULT_TABLE table=test-table [id=b331f88e7bc04ba6818d786d379d0d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:52.984285 26312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a56b77c77a64d98be7df535649a6ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:52.987115 26270 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Bootstrap starting.
I20260926 16:39:52.988035 26270 tablet_bootstrap.cc:654] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:52.988181 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.983133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.11.158.130:39611)
0926 16:39:52.983518 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.983534 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.983557 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:39:52.983631 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.983635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.983821 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.983986 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:52.984407 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.985480 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.986680 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.987624 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.987763 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.987907 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.987966 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.988050 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":91,"threads_started":1}
I20260926 16:39:52.988303 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.982442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:42829 (local address 127.0.0.1:36296)
0926 16:39:52.982574 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.982590 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.982668 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.983528 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.983531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.983549 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:52.983671 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.983678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.985363 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.985366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.986668 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.986679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.987837 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.987840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.987957 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.987994 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.988054 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.988137 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":65}
I20260926 16:39:52.988173 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.982726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39611 (local address 127.0.0.1:45690)
0926 16:39:52.983235 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:52.983248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:52.983335 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:52.984148 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:52.984151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:52.984168 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:52.984255 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.984261 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.985611 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.985617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.986507 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:52.986516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.987748 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.987752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:52.987835 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.987875 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 16:39:52.987943 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 16:39:52.988028 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260926 16:39:52.989280 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:52.982630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.11.158.131:42829)
0926 16:39:52.983009 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:52.983022 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:52.983047 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:39:52.983125 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:52.983131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:52.983355 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:52.983542 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:52.983923 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.985211 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.986836 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:52.987698 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:52.988847 (+ 1149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:52.988974 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:52.989028 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 16:39:52.989120 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":206,"threads_started":1}
I20260926 16:39:52.989694 26270 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: No bootstrap required, opened a new log
I20260926 16:39:52.989802 26270 ts_tablet_manager.cc:1397] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:39:52.990239 26270 raft_consensus.cc:348] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.990366 26270 raft_consensus.cc:374] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:52.990422 26270 raft_consensus.cc:729] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53abe034f46a4db1bb870546b0d73e01, State: Initialized, Role: FOLLOWER
I20260926 16:39:52.990727 26270 consensus_queue.cc:260] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:52.991112 26270 ts_tablet_manager.cc:1428] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:39:52.991474 26450 tablet_service.cc:1467] Processing CreateTablet for tablet 7a56b77c77a64d98be7df535649a6ddd (DEFAULT_TABLE table=test-table [id=b331f88e7bc04ba6818d786d379d0d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:52.991940 26582 tablet_service.cc:1467] Processing CreateTablet for tablet 7a56b77c77a64d98be7df535649a6ddd (DEFAULT_TABLE table=test-table [id=b331f88e7bc04ba6818d786d379d0d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:39:52.992468 26450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a56b77c77a64d98be7df535649a6ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:52.992905 26582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a56b77c77a64d98be7df535649a6ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:39:52.999759 26662 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Bootstrap starting.
I20260926 16:39:52.999653 26661 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Bootstrap starting.
I20260926 16:39:53.002393 26662 tablet_bootstrap.cc:654] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:53.002434 26661 tablet_bootstrap.cc:654] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Neither blocks nor log segments found. Creating new log.
I20260926 16:39:53.003125 26662 log.cc:824] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:53.003125 26661 log.cc:824] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:53.005050 26661 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: No bootstrap required, opened a new log
I20260926 16:39:53.005055 26662 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: No bootstrap required, opened a new log
I20260926 16:39:53.005244 26662 ts_tablet_manager.cc:1397] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:39:53.005517 26661 ts_tablet_manager.cc:1397] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:39:53.008081 26662 raft_consensus.cc:348] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.008325 26662 raft_consensus.cc:374] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:53.008185 26661 raft_consensus.cc:348] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.008402 26662 raft_consensus.cc:729] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf5553f714a4cc5a01827f7cc0c66b2, State: Initialized, Role: FOLLOWER
I20260926 16:39:53.008445 26661 raft_consensus.cc:374] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:39:53.008503 26661 raft_consensus.cc:729] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4c7dd604a64e508d66349cdff026df, State: Initialized, Role: FOLLOWER
I20260926 16:39:53.008849 26662 consensus_queue.cc:260] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.008903 26661 consensus_queue.cc:260] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.009495 26516 heartbeater.cc:503] Master 127.11.158.190:34959 was elected leader, sending a full tablet report...
I20260926 16:39:53.009512 26648 heartbeater.cc:503] Master 127.11.158.190:34959 was elected leader, sending a full tablet report...
I20260926 16:39:53.009837 26661 ts_tablet_manager.cc:1428] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 16:39:53.009837 26662 ts_tablet_manager.cc:1428] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260926 16:39:53.062744 26379 tablet.cc:2369] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:39:53.111459 26665 raft_consensus.cc:482] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:39:53.111696 26665 raft_consensus.cc:504] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.113291 26665 leader_election.cc:288] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829), 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609)
I20260926 16:39:53.117254 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.113652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:41137 (local address 127.11.158.129:36609)
0926 16:39:53.113919 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:53.113922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:53.114086 (+ 164us) server_negotiation.cc:407] Connection header received
0926 16:39:53.114199 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:53.114202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:53.114262 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:53.114382 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:53.114675 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.115201 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.116225 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.116677 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.116808 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:53.116959 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:53.117066 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 16:39:53.117137 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":89,"threads_started":1}
I20260926 16:39:53.117251 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.113775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:60149 (local address 127.11.158.131:42829)
0926 16:39:53.113902 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:53.113906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:53.113922 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:39:53.113976 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:53.113979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:53.114055 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:53.114132 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:39:53.114434 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.115054 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.116160 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.116674 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.116807 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:53.116864 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:53.117053 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 16:39:53.117119 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:39:53.117254 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.113507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:42829 (local address 127.11.158.130:60149)
0926 16:39:53.113655 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.113671 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:53.113769 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:53.114165 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:53.114170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:53.114191 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:53.114321 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:53.114331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.115166 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.115169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:53.116033 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:53.116042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.116797 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.116800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:53.116905 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:53.116931 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:39:53.117029 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 16:39:53.117106 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260926 16:39:53.117262 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.113604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36609 (local address 127.11.158.130:41137)
0926 16:39:53.113996 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.114009 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:53.114114 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:53.114420 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:53.114423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:53.114440 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:39:53.114562 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:53.114569 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.115316 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.115318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:53.116079 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:53.116111 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.116807 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.116811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:53.116922 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:53.116954 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 16:39:53.117049 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 16:39:53.117118 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":29,"thread_start_us":100,"threads_started":1}
I20260926 16:39:53.117986 26602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a56b77c77a64d98be7df535649a6ddd" candidate_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4c7dd604a64e508d66349cdff026df" is_pre_election: true
I20260926 16:39:53.117981 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a56b77c77a64d98be7df535649a6ddd" candidate_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53abe034f46a4db1bb870546b0d73e01" is_pre_election: true
I20260926 16:39:53.118288 26330 raft_consensus.cc:2454] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdf5553f714a4cc5a01827f7cc0c66b2 in term 0.
I20260926 16:39:53.118288 26602 raft_consensus.cc:2454] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdf5553f714a4cc5a01827f7cc0c66b2 in term 0.
I20260926 16:39:53.118765 26405 leader_election.cc:302] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a4c7dd604a64e508d66349cdff026df, 53abe034f46a4db1bb870546b0d73e01, fdf5553f714a4cc5a01827f7cc0c66b2; no voters:
I20260926 16:39:53.119064 26665 raft_consensus.cc:2781] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:39:53.119197 26665 raft_consensus.cc:482] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:39:53.119264 26665 raft_consensus.cc:3037] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:53.120280 26665 raft_consensus.cc:504] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.120644 26665 leader_election.cc:288] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [CANDIDATE]: Term 1 election: Requested vote from peers 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829), 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609)
I20260926 16:39:53.121001 26602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a56b77c77a64d98be7df535649a6ddd" candidate_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4c7dd604a64e508d66349cdff026df"
I20260926 16:39:53.121007 26330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a56b77c77a64d98be7df535649a6ddd" candidate_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53abe034f46a4db1bb870546b0d73e01"
I20260926 16:39:53.121127 26330 raft_consensus.cc:3037] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:53.121127 26602 raft_consensus.cc:3037] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:39:53.122354 26330 raft_consensus.cc:2454] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdf5553f714a4cc5a01827f7cc0c66b2 in term 1.
I20260926 16:39:53.122354 26602 raft_consensus.cc:2454] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdf5553f714a4cc5a01827f7cc0c66b2 in term 1.
I20260926 16:39:53.122658 26404 leader_election.cc:302] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4c7dd604a64e508d66349cdff026df, fdf5553f714a4cc5a01827f7cc0c66b2; no voters:
I20260926 16:39:53.122840 26665 raft_consensus.cc:2781] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:39:53.123080 26665 raft_consensus.cc:686] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 LEADER]: Becoming Leader. State: Replica: fdf5553f714a4cc5a01827f7cc0c66b2, State: Running, Role: LEADER
I20260926 16:39:53.123394 26665 consensus_queue.cc:237] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.126436 26190 catalog_manager.cc:5495] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130). New cstate: current_term: 1 leader_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260926 16:39:53.147468 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.144421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.11.158.130:39611)
0926 16:39:53.144546 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:53.144549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:53.144563 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:39:53.144632 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:53.144634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:53.144687 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:53.144779 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:39:53.144974 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.145464 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.146246 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:53.146446 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:53.146520 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:53.146751 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:39:53.147152 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:39:53.147240 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:53.147298 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 16:39:53.147345 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:39:53.153128 26602 raft_consensus.cc:1261] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 1 FOLLOWER]: Refusing update from remote peer fdf5553f714a4cc5a01827f7cc0c66b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:53.153126 26330 raft_consensus.cc:1261] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Refusing update from remote peer fdf5553f714a4cc5a01827f7cc0c66b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:39:53.153813 26669 consensus_queue.cc:1035] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:39:53.154017 26665 consensus_queue.cc:1035] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:39:53.158910 26672 mvcc.cc:205] Tried to move back new op lower bound from 7333645488747917312 to 7333645488635867136. Current Snapshot: MvccSnapshot[applied={T|T < 7333645488747917312}]
I20260926 16:39:53.162313 26674 mvcc.cc:205] Tried to move back new op lower bound from 7333645488747917312 to 7333645488635867136. Current Snapshot: MvccSnapshot[applied={T|T < 7333645488747917312}]
I20260926 16:39:53.167526 26189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:39:53.167703 26189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:39:53.169974 26189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Sending DeleteTablet for 3 replicas of tablet 7a56b77c77a64d98be7df535649a6ddd
I20260926 16:39:53.170622 26582 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a56b77c77a64d98be7df535649a6ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:53 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:36296
I20260926 16:39:53.170912 26450 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a56b77c77a64d98be7df535649a6ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:53 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:45690
I20260926 16:39:53.171196 26682 tablet_replica.cc:330] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: stopping tablet replica
I20260926 16:39:53.171262 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 26228
I20260926 16:39:53.171308 26312 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a56b77c77a64d98be7df535649a6ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:39:53 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:56004
I20260926 16:39:53.172165 26683 tablet_replica.cc:330] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: stopping tablet replica
I20260926 16:39:53.172497 26683 raft_consensus.cc:2229] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:39:53.172817 26683 raft_consensus.cc:2258] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:39:53.172919 26682 raft_consensus.cc:2229] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:39:53.173224 26682 raft_consensus.cc:2258] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:39:53.174571 26682 ts_tablet_manager.cc:1905] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:53.177479 26683 ts_tablet_manager.cc:1905] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:53.177584 26682 ts_tablet_manager.cc:1918] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:53.177691 26682 log.cc:1196] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/7a56b77c77a64d98be7df535649a6ddd
I20260926 16:39:53.178156 26682 ts_tablet_manager.cc:1939] T 7a56b77c77a64d98be7df535649a6ddd P 0a4c7dd604a64e508d66349cdff026df: Deleting consensus metadata
W20260926 16:39:53.179112 26176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:36609 (error 108)
I20260926 16:39:53.179615 26175 catalog_manager.cc:4841] TS 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829): tablet 7a56b77c77a64d98be7df535649a6ddd (table test-table [id=b331f88e7bc04ba6818d786d379d0d4a]) successfully deleted
I20260926 16:39:53.180197 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 26383
I20260926 16:39:53.180444 26683 ts_tablet_manager.cc:1918] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:53.180544 26683 log.cc:1196] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/7a56b77c77a64d98be7df535649a6ddd
I20260926 16:39:53.181490 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.180512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:36609 (local address 127.0.0.1:56010)
0926 16:39:53.181356 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.181420 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:36609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":765}
W20260926 16:39:53.181723 26176 catalog_manager.cc:4584] TS 53abe034f46a4db1bb870546b0d73e01 (127.11.158.129:36609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a56b77c77a64d98be7df535649a6ddd: Network error: Client connection negotiation failed: client connection to 127.11.158.129:36609: connect: Connection refused (error 111)
I20260926 16:39:53.186944 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 26519
I20260926 16:39:53.187253 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.186943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39611 (local address 127.0.0.1:45702)
0926 16:39:53.187098 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.187161 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.130:39611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 16:39:53.191938 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 26157
I20260926 16:39:53.198060 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34959
--webserver_interface=127.11.158.190
--webserver_port=42811
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:34959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:39:53.309000 26685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:53.309274 26685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:53.309317 26685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:53.313421 26685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:39:53.313477 26685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:53.313498 26685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:53.313534 26685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:39:53.313567 26685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:39:53.318862 26685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:34959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:34959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:53.319926 26685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:53.320982 26685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:53.329614 26693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:53.329679 26691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:53.329627 26690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:53.329927 26685 server_base.cc:1034] running on GCE node
I20260926 16:39:53.330689 26685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:53.331799 26685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:53.332954 26685 hybrid_clock.cc:648] HybridClock initialized: now 1790440793332929 us; error 37 us; skew 500 ppm
Sep 26 16:39:53 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:53.340397 26685 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:39:53.341367 26685 webserver.cc:460] Webserver started at http://127.11.158.190:42811/ using document root <none> and password file <none>
I20260926 16:39:53.341910 26685 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:53.341970 26685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:53.345775 26685 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 16:39:53.347869 26700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:53.349248 26685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:39:53.349419 26685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "9975002eb7a84cd6a4114a47515428cf"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "127390ac2f105bfa53a6d0d958a7552e"
server_key_iv: "0f7f67300210b203fcba976046adcef9"
server_key_version: "encryptionkey@0"
I20260926 16:39:53.349983 26685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:53.374126 26685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:53.377674 26685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:53.377892 26685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:53.385394 26685 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:34959
I20260926 16:39:53.385700 26752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:34959 every 8 connection(s)
I20260926 16:39:53.386600 26685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:39:53.390789 26753 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:39:53.392457 26753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Bootstrap starting.
I20260926 16:39:53.394706 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 26685
I20260926 16:39:53.395141 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:36609
--local_ip_for_outbound_sockets=127.11.158.129
--tserver_master_addrs=127.11.158.190:34959
--webserver_port=40861
--webserver_interface=127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:39:53.409947 26753 log.cc:824] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:53.422348 26753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:39:53.422796 26753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf: Bootstrap complete.
I20260926 16:39:53.427368 26753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:53.427796 26753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9975002eb7a84cd6a4114a47515428cf, State: Initialized, Role: FOLLOWER
I20260926 16:39:53.428329 26753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:53.428524 26753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:39:53.428617 26753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:39:53.428743 26753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 2 FOLLOWER]: Advancing to term 3
I20260926 16:39:53.430851 26753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:53.431216 26753 leader_election.cc:302] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9975002eb7a84cd6a4114a47515428cf; no voters:
I20260926 16:39:53.431566 26753 leader_election.cc:288] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 16:39:53.431841 26757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 3 FOLLOWER]: Leader election won for term 3
I20260926 16:39:53.432255 26757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [term 3 LEADER]: Becoming Leader. State: Replica: 9975002eb7a84cd6a4114a47515428cf, State: Running, Role: LEADER
I20260926 16:39:53.432685 26757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } }
I20260926 16:39:53.433007 26753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:39:53.434214 26758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9975002eb7a84cd6a4114a47515428cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } } }
I20260926 16:39:53.434345 26758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:53.434722 26759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9975002eb7a84cd6a4114a47515428cf. Latest consensus state: current_term: 3 leader_uuid: "9975002eb7a84cd6a4114a47515428cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9975002eb7a84cd6a4114a47515428cf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 34959 } } }
I20260926 16:39:53.434857 26759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9975002eb7a84cd6a4114a47515428cf [sys.catalog]: This master's current role is: LEADER
I20260926 16:39:53.435177 26766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:39:53.440825 26766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:39:53.441470 26766 catalog_manager.cc:1259] Loaded cluster ID: 01dee4bf79304369ab9b79ac42d69934
I20260926 16:39:53.441533 26766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 16:39:53.442498 26766 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7efdf78a19ad google::LogMessage::Fail() at ??:0
@ 0x7efdf78a488a google::LogMessage::SendToLog() at ??:0
@ 0x7efdf78a14de google::LogMessage::Flush() at ??:0
@ 0x7efdf78a2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7efe021e0fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7efe021e1d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7efe021de579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7efe02222640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7efdf9af33d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7efdf9af3c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7efdf9af53f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7efdf9ae4512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7efdfb1286db start_thread at ??:0
@ 0x7efdf639f61f clone at ??:0
W20260926 16:39:53.545814 26755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:53.546111 26755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:53.546187 26755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:53.550403 26755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:53.550490 26755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:53.550578 26755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:39:53.558140 26755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:36609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=40861
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:53.559595 26755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:53.560894 26755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:53.572996 26794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:53.573175 26795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:53.573374 26797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:53.575785 26755 server_base.cc:1034] running on GCE node
I20260926 16:39:53.576364 26755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:53.577247 26755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:53.578452 26755 hybrid_clock.cc:648] HybridClock initialized: now 1790440793578419 us; error 50 us; skew 500 ppm
Sep 26 16:39:53 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:53.582304 26755 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:39:53.583403 26755 webserver.cc:460] Webserver started at http://127.11.158.129:40861/ using document root <none> and password file <none>
I20260926 16:39:53.584039 26755 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:53.584141 26755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:53.589035 26755 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:39:53.591432 26804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:53.592954 26755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:53.593101 26755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "53abe034f46a4db1bb870546b0d73e01"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "663bce7310dca7fead9edad24ffac14b"
server_key_iv: "47a3530ab73a2a2a17b88f43f615110d"
server_key_version: "encryptionkey@0"
I20260926 16:39:53.593686 26755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:53.613073 26755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:53.616766 26755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:53.617005 26755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:53.617684 26755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:53.618985 26811 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:39:53.623507 26755 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:39:53.623600 26755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260926 16:39:53.623674 26755 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:39:53.625488 26755 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:39:53.625554 26755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260926 16:39:53.625917 26811 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Bootstrap starting.
I20260926 16:39:53.637372 26755 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:36609
I20260926 16:39:53.638747 26755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:39:53.639998 26918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:36609 every 8 connection(s)
I20260926 16:39:53.642582 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 26755
I20260926 16:39:53.643203 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:39611
--local_ip_for_outbound_sockets=127.11.158.130
--tserver_master_addrs=127.11.158.190:34959
--webserver_port=38593
--webserver_interface=127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:39:53.649478 26811 log.cc:824] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Log is configured to *not* fsync() on all Append() calls
I20260926 16:39:53.659396 26811 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:39:53.659932 26811 tablet_bootstrap.cc:492] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Bootstrap complete.
I20260926 16:39:53.661497 26811 ts_tablet_manager.cc:1397] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Time spent bootstrapping tablet: real 0.036s user 0.018s sys 0.015s
I20260926 16:39:53.665091 26811 raft_consensus.cc:348] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.665688 26811 raft_consensus.cc:729] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53abe034f46a4db1bb870546b0d73e01, State: Initialized, Role: FOLLOWER
I20260926 16:39:53.666224 26811 consensus_queue.cc:260] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:53.667306 26811 ts_tablet_manager.cc:1428] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260926 16:39:53.715807 26921 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:39:53.641174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:36043)
0926 16:39:53.641887 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.641948 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:53.643025 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:53.714971 (+ 71946us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":246,"threads_started":1}
W20260926 16:39:53.716516 26919 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:34959 (0 consecutive failures): Network error: Failed to ping master at 127.11.158.190:34959: Client connection negotiation failed: client connection to 127.11.158.190:34959: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 16:39:53.717285 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.717012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:48153)
0926 16:39:53.717146 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.717218 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 16:39:53.718024 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.717811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:39181)
0926 16:39:53.717919 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.717973 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:39:53.718226 26919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:39:53.769778 26922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:53.770117 26922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:53.770219 26922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:53.774731 26922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:53.774838 26922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:53.774940 26922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:39:53.780413 26922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:39611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=38593
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:53.781518 26922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:53.782536 26922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:53.792524 26932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:53.792619 26922 server_base.cc:1034] running on GCE node
W20260926 16:39:53.792763 26929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:53.792654 26930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:53.793206 26922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:53.793826 26922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:53.795048 26922 hybrid_clock.cc:648] HybridClock initialized: now 1790440793795015 us; error 44 us; skew 500 ppm
Sep 26 16:39:53 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:53.797922 26922 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:39:53.798861 26922 webserver.cc:460] Webserver started at http://127.11.158.130:38593/ using document root <none> and password file <none>
I20260926 16:39:53.799448 26922 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:53.799537 26922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:53.804517 26922 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:53.807214 26939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:53.808679 26922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:39:53.808830 26922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "fdf5553f714a4cc5a01827f7cc0c66b2"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "5b669ecd9ea8d3c7ef6bab0dda7a5de0"
server_key_iv: "95a0cb5b1f54e84730588cb7480985f2"
server_key_version: "encryptionkey@0"
I20260926 16:39:53.809448 26922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:53.822100 26922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:53.825698 26922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:53.825941 26922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:53.826583 26922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:53.827741 26946 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 16:39:53.831686 26946 ts_tablet_manager.cc:1852] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 16:39:53.832109 26946 ts_tablet_manager.cc:1905] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:39:53.836231 26946 ts_tablet_manager.cc:1918] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:39:53.836412 26946 log.cc:1196] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/7a56b77c77a64d98be7df535649a6ddd
I20260926 16:39:53.836820 26946 ts_tablet_manager.cc:1939] T 7a56b77c77a64d98be7df535649a6ddd P fdf5553f714a4cc5a01827f7cc0c66b2: Deleting consensus metadata
I20260926 16:39:53.837167 26922 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 16:39:53.837261 26922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260926 16:39:53.837350 26922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:53.837388 26922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:53.846429 26922 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:39611
I20260926 16:39:53.846467 27053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:39611 every 8 connection(s)
I20260926 16:39:53.847319 26922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:39:53.850812 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.849447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:35661)
0926 16:39:53.849943 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.850067 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":113,"threads_started":1}
W20260926 16:39:53.851262 27054 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:34959 (0 consecutive failures): Network error: Failed to ping master at 127.11.158.190:34959: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
I20260926 16:39:53.851853 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 26922
I20260926 16:39:53.852058 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.851734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:52589)
0926 16:39:53.851900 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.851991 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 16:39:53.852402 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:42829
--local_ip_for_outbound_sockets=127.11.158.131
--tserver_master_addrs=127.11.158.190:34959
--webserver_port=34869
--webserver_interface=127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:39:53.852862 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:53.852652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:45973)
0926 16:39:53.852763 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:53.852811 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 16:39:53.853113 27054 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:39:53.965515 27057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:39:53.965834 27057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:39:53.965916 27057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:39:53.970105 27057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:39:53.970206 27057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:39:53.970297 27057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:39:53.976960 27057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:42829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=34869
--tserver_master_addrs=127.11.158.190:34959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:39:53.978152 27057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:39:53.979208 27057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:39:53.989218 27062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:53.989230 27065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:39:53.989238 27063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:39:53.989444 27057 server_base.cc:1034] running on GCE node
I20260926 16:39:53.990002 27057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:39:53.990597 27057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:39:53.991849 27057 hybrid_clock.cc:648] HybridClock initialized: now 1790440793991825 us; error 27 us; skew 500 ppm
Sep 26 16:39:53 dist-test-slave-p39x krb5kdc[25611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:39:53.995100 27057 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:39:53.996029 27057 webserver.cc:460] Webserver started at http://127.11.158.131:34869/ using document root <none> and password file <none>
I20260926 16:39:53.996708 27057 fs_manager.cc:359] Metadata directory not provided
I20260926 16:39:53.996821 27057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:39:54.001066 27057 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 16:39:54.004047 27072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:54.005589 27057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:39:54.005769 27057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "0a4c7dd604a64e508d66349cdff026df"
format_stamp: "Formatted at 2026-09-26 16:39:51 on dist-test-slave-p39x"
server_key: "d6d9712dc950fbb1c83b7940caf4b45c"
server_key_iv: "ed546a196979f83a622d3bd68b1555d7"
server_key_version: "encryptionkey@0"
I20260926 16:39:54.006394 27057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:39:54.019299 27057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:39:54.022418 27057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:39:54.022650 27057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:39:54.023283 27057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:39:54.024150 27057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:39:54.024222 27057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:39:54.024292 27057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:39:54.024377 27057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 16:39:54.033916 27057 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:42829
I20260926 16:39:54.033939 27185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:42829 every 8 connection(s)
I20260926 16:39:54.034966 27057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:39:54.039155 27188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.037475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:47969)
0926 16:39:54.037975 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:54.038415 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":125,"threads_started":1}
I20260926 16:39:54.039613 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 27057
W20260926 16:39:54.039753 27186 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:34959 (0 consecutive failures): Network error: Failed to ping master at 127.11.158.190:34959: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
I20260926 16:39:54.040539 27188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.040236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:43379)
0926 16:39:54.040400 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:54.040463 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 16:39:54.041342 27188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.041104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:41159)
0926 16:39:54.041237 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:54.041287 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260926 16:39:54.041571 27186 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:39:54.719276 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.718835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:49139)
0926 16:39:54.719101 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:54.719165 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260926 16:39:54.854259 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.853817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:49329)
0926 16:39:54.854100 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:54.854165 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":72,"threads_started":1}
I20260926 16:39:54.936967 27192 raft_consensus.cc:482] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:39:54.937211 27192 raft_consensus.cc:504] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:54.938432 27192 leader_election.cc:288] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829), fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611)
Sep 26 16:39:54 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.130@KRBTEST.COM
Sep 26 16:39:54 dist-test-slave-p39x krb5kdc[25611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440793, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.131@KRBTEST.COM
I20260926 16:39:54.953737 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.938738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39611 (local address 127.11.158.129:57123)
0926 16:39:54.939255 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:54.939267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:54.939355 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:54.941537 (+ 2182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:54.941539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:54.941700 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:54.941977 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:54.941992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.943306 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.943309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:54.944449 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:54.944459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.945186 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.945189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:54.945371 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:54.946098 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:54.946110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:54.947720 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:54.950369 (+ 2649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:54.950377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:54.950392 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:54.950879 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:54.951356 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:54.951360 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:54.951362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:54.951591 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:54.952518 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:54.952521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:54.952913 (+ 392us) client_negotiation.cc:769] Sending connection context
0926 16:39:54.953138 (+ 225us) client_negotiation.cc:240] Negotiation successful
0926 16:39:54.953578 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":392,"spinlock_wait_cycles":50304,"thread_start_us":79,"threads_started":1}
I20260926 16:39:54.953771 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.938935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:57123 (local address 127.11.158.130:39611)
0926 16:39:54.939271 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:54.939287 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:54.939330 (+ 43us) server_negotiation.cc:407] Connection header received
0926 16:39:54.940226 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:54.940232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:54.940709 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:54.941051 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:54.942202 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.943132 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.944564 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.945040 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.945225 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:54.947974 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:54.947980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:54.947992 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:54.948020 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:54.950200 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:54.951042 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:54.951049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:54.951055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:54.951210 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:54.951736 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:54.951740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:54.951742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:54.952371 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:54.952515 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:54.953280 (+ 765us) server_negotiation.cc:299] Negotiation successful
0926 16:39:54.953590 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":75,"threads_started":1}
I20260926 16:39:54.953778 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.938888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:60951 (local address 127.11.158.131:42829)
0926 16:39:54.939271 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0926 16:39:54.939287 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:39:54.939330 (+ 43us) server_negotiation.cc:407] Connection header received
0926 16:39:54.940206 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:39:54.940216 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:39:54.940709 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:39:54.941051 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:39:54.942165 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.943132 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.944594 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.945034 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.945203 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:54.948265 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:39:54.948275 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:39:54.948290 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:39:54.948327 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:39:54.950200 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:54.951042 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:54.951049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:54.951055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:54.951210 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:39:54.951736 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:39:54.951740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:39:54.951742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:39:54.952372 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:39:54.952515 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 16:39:54.953280 (+ 765us) server_negotiation.cc:299] Negotiation successful
0926 16:39:54.953581 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":72,"threads_started":1}
I20260926 16:39:54.953737 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:54.938735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:42829 (local address 127.11.158.129:60951)
0926 16:39:54.938873 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:54.938891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:39:54.939013 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:39:54.941147 (+ 2134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:39:54.941152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:39:54.941523 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:39:54.941977 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:54.941992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.943293 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.943298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:54.944448 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:39:54.944463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:39:54.945158 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:39:54.945162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:39:54.945359 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:39:54.946040 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:39:54.946056 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:39:54.948060 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:39:54.950369 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:54.950377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:54.950392 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:54.950879 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:54.951356 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:39:54.951360 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:39:54.951362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:39:54.951591 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:39:54.952505 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:39:54.952512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:39:54.952913 (+ 401us) client_negotiation.cc:769] Sending connection context
0926 16:39:54.953138 (+ 225us) client_negotiation.cc:240] Negotiation successful
0926 16:39:54.953526 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":249,"spinlock_wait_cycles":41344}
W20260926 16:39:54.956826 26805 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
W20260926 16:39:54.957026 26806 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
I20260926 16:39:54.957129 26806 leader_election.cc:302] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53abe034f46a4db1bb870546b0d73e01; no voters: 0a4c7dd604a64e508d66349cdff026df, fdf5553f714a4cc5a01827f7cc0c66b2
I20260926 16:39:54.957412 27192 raft_consensus.cc:2726] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:39:55.042598 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:55.042197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:37313)
0926 16:39:55.042445 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:55.042503 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
I20260926 16:39:55.720700 27197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:55.720135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:46629)
0926 16:39:55.720499 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:55.720598 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":87,"threads_started":1}
I20260926 16:39:55.855566 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:55.855098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:51655)
0926 16:39:55.855387 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:55.855453 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":76,"threads_started":1}
I20260926 16:39:56.043892 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:56.043438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:37057)
0926 16:39:56.043738 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:56.043791 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":145,"threads_started":1}
I20260926 16:39:56.722004 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:56.721581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:53061)
0926 16:39:56.721843 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:56.721905 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
I20260926 16:39:56.857059 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:56.856555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:56693)
0926 16:39:56.856884 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:56.856962 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":77,"threads_started":1}
I20260926 16:39:57.045338 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:57.044839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:36065)
0926 16:39:57.045166 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:57.045235 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":96,"threads_started":1}
I20260926 16:39:57.150960 27203 raft_consensus.cc:482] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:39:57.151136 27203 raft_consensus.cc:504] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:39:57.151597 27203 leader_election.cc:288] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829), fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611)
W20260926 16:39:57.152612 26805 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
W20260926 16:39:57.152726 26806 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
I20260926 16:39:57.152791 26806 leader_election.cc:302] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53abe034f46a4db1bb870546b0d73e01; no voters: 0a4c7dd604a64e508d66349cdff026df, fdf5553f714a4cc5a01827f7cc0c66b2
I20260926 16:39:57.153004 27203 raft_consensus.cc:2726] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:39:57.723302 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:57.722855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:46657)
0926 16:39:57.723137 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:57.723203 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":90,"threads_started":1}
I20260926 16:39:57.858361 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:57.857946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:53083)
0926 16:39:57.858197 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:57.858256 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260926 16:39:58.046664 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:58.046189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:43101)
0926 16:39:58.046478 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:58.046551 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":81,"threads_started":1}
I20260926 16:39:58.725085 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:58.724585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:58285)
0926 16:39:58.724914 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:58.724982 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":74,"threads_started":1}
I20260926 16:39:58.859673 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:58.859245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:48767)
0926 16:39:58.859511 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:58.859574 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260926 16:39:59.048054 27209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:59.047607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:49889)
0926 16:39:59.047889 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:59.047954 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":91,"threads_started":1}
I20260926 16:39:59.726557 27210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:59.726086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:51383)
0926 16:39:59.726372 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:59.726441 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
I20260926 16:39:59.861066 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:39:59.860499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:57699)
0926 16:39:59.860874 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0926 16:39:59.860953 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260926 16:40:00.049463 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:00.049028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:56325)
0926 16:40:00.049296 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:00.049363 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
I20260926 16:40:00.280588 27213 raft_consensus.cc:482] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:00.280714 27213 raft_consensus.cc:504] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:40:00.281152 27213 leader_election.cc:288] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829), fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611)
W20260926 16:40:00.281966 26806 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
W20260926 16:40:00.282068 26805 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
I20260926 16:40:00.282136 26805 leader_election.cc:302] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53abe034f46a4db1bb870546b0d73e01; no voters: 0a4c7dd604a64e508d66349cdff026df, fdf5553f714a4cc5a01827f7cc0c66b2
I20260926 16:40:00.282267 27213 raft_consensus.cc:2726] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:40:00.727900 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:00.727441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:38791)
0926 16:40:00.727737 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:00.727798 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
I20260926 16:40:00.862380 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:00.861893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:35911)
0926 16:40:00.862223 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:00.862285 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":94,"threads_started":1}
I20260926 16:40:01.050776 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:01.050264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:37203)
0926 16:40:01.050599 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:01.050675 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":74,"threads_started":1}
I20260926 16:40:01.729240 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:01.728792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:54439)
0926 16:40:01.729079 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:01.729142 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
I20260926 16:40:01.863762 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:01.863264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:58079)
0926 16:40:01.863578 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:01.863659 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":105,"threads_started":1}
I20260926 16:40:02.052306 27219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:02.051814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:47719)
0926 16:40:02.052141 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:02.052206 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":89,"threads_started":1}
I20260926 16:40:02.730381 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:02.729978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:55893)
0926 16:40:02.730230 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:02.730287 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":71,"threads_started":1}
I20260926 16:40:02.865110 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:02.864681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:38419)
0926 16:40:02.864946 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:02.865011 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260926 16:40:03.053746 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:03.053306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:52715)
0926 16:40:03.053595 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:03.053653 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":85,"threads_started":1}
I20260926 16:40:03.216750 27223 raft_consensus.cc:482] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:03.216897 27223 raft_consensus.cc:504] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:40:03.217386 27223 leader_election.cc:288] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829), fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611)
W20260926 16:40:03.218179 26805 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
W20260926 16:40:03.218288 26806 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
I20260926 16:40:03.218400 26806 leader_election.cc:302] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53abe034f46a4db1bb870546b0d73e01; no voters: 0a4c7dd604a64e508d66349cdff026df, fdf5553f714a4cc5a01827f7cc0c66b2
I20260926 16:40:03.218600 27223 raft_consensus.cc:2726] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:40:03.731801 27224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:03.731353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:33235)
0926 16:40:03.731633 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:03.731699 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
I20260926 16:40:03.866518 27225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:03.866069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:35775)
0926 16:40:03.866354 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:03.866423 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
I20260926 16:40:04.055140 27226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:04.054632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:36617)
0926 16:40:04.054970 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:04.055038 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":85,"threads_started":1}
I20260926 16:40:04.733208 27227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:04.732778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:57375)
0926 16:40:04.733047 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:04.733111 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260926 16:40:04.867712 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:04.867312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:55105)
0926 16:40:04.867557 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:04.867619 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260926 16:40:05.056474 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:05.055997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:57215)
0926 16:40:05.056261 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:05.056313 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":97,"threads_started":1}
I20260926 16:40:05.734598 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:05.734124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:58489)
0926 16:40:05.734426 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:05.734491 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":98,"threads_started":1}
I20260926 16:40:05.869189 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:05.868618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:59107)
0926 16:40:05.868942 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:05.869051 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":81,"threads_started":1}
I20260926 16:40:06.057753 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:06.057267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:52217)
0926 16:40:06.057577 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:06.057653 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":83,"threads_started":1}
I20260926 16:40:06.735785 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:06.735385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:45323)
0926 16:40:06.735633 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:06.735691 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260926 16:40:06.870440 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:06.870039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:43755)
0926 16:40:06.870283 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:06.870343 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260926 16:40:07.059068 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:07.058624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:50117)
0926 16:40:07.058904 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:07.058969 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260926 16:40:07.452813 27236 raft_consensus.cc:482] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:07.452950 27236 raft_consensus.cc:504] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a4c7dd604a64e508d66349cdff026df" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 42829 } } peers { permanent_uuid: "fdf5553f714a4cc5a01827f7cc0c66b2" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39611 } } peers { permanent_uuid: "53abe034f46a4db1bb870546b0d73e01" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 36609 } }
I20260926 16:40:07.453343 27236 leader_election.cc:288] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829), fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611)
W20260926 16:40:07.454077 26805 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fdf5553f714a4cc5a01827f7cc0c66b2 (127.11.158.130:39611): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
W20260926 16:40:07.454169 26806 leader_election.cc:341] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a4c7dd604a64e508d66349cdff026df (127.11.158.131:42829): Not found: Tablet not found: 7a56b77c77a64d98be7df535649a6ddd
I20260926 16:40:07.454262 26806 leader_election.cc:302] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53abe034f46a4db1bb870546b0d73e01; no voters: 0a4c7dd604a64e508d66349cdff026df, fdf5553f714a4cc5a01827f7cc0c66b2
I20260926 16:40:07.454418 27236 raft_consensus.cc:2726] T 7a56b77c77a64d98be7df535649a6ddd P 53abe034f46a4db1bb870546b0d73e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:40:07.737217 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:07.736743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:41051)
0926 16:40:07.737050 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:07.737109 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":93,"threads_started":1}
I20260926 16:40:07.871984 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:07.871339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:57237)
0926 16:40:07.871813 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:07.871884 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":91,"threads_started":1}
I20260926 16:40:08.060597 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:08.059907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.131:57399)
0926 16:40:08.060224 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:08.060294 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":76,"threads_started":1}
I20260926 16:40:08.738550 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:08.738173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.129:45763)
0926 16:40:08.738406 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:08.738458 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
I20260926 16:40:08.873456 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:08.873036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:34959 (local address 127.11.158.130:53347)
0926 16:40:08.873297 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:08.873358 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
I20260926 16:40:09.044015 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 26755
I20260926 16:40:09.050025 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 26922
I20260926 16:40:09.055449 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 27057
2026-09-26T16:40:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19206 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:40:09 dist-test-slave-p39x krb5kdc[27244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:40:09 dist-test-slave-p39x krb5kdc[27244](info): set up 2 sockets
Sep 26 16:40:09 dist-test-slave-p39x krb5kdc[27244](info): commencing operation
krb5kdc: starting...
W20260926 16:40:10.117430 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:40:10 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:40:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:40:10.240090 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38299
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:40:10.348697 27257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:10.348965 27257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:10.349035 27257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:10.353044 27257 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 16:40:10.353118 27257 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 16:40:10.353145 27257 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 16:40:10.353183 27257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:40:10.353225 27257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:10.353286 27257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:10.353322 27257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:40:10.353358 27257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:40:10.358681 27257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:10.359809 27257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:10.360877 27257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:10.370007 27262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:10.369985 27257 server_base.cc:1034] running on GCE node
W20260926 16:40:10.369990 27263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:10.369949 27265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:10.370765 27257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:10.371951 27257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:10.373118 27257 hybrid_clock.cc:648] HybridClock initialized: now 1790440810373098 us; error 31 us; skew 500 ppm
Sep 26 16:40:10 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440810, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:10.375914 27257 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:40:10.376816 27257 webserver.cc:460] Webserver started at http://127.11.158.190:38389/ using document root <none> and password file <none>
I20260926 16:40:10.377338 27257 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:10.377396 27257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:10.377614 27257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:40:10.379582 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "03a2efcf9d9a46f096a2e9434a1845f9"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "6c6e299453480fbfb2f2877b21dee43c"
server_key_iv: "61b11f6774c7c5f25cf0fa751b50bf8c"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.380100 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "03a2efcf9d9a46f096a2e9434a1845f9"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "6c6e299453480fbfb2f2877b21dee43c"
server_key_iv: "61b11f6774c7c5f25cf0fa751b50bf8c"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.383673 27257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:40:10.386047 27272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:10.387357 27257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:40:10.387457 27257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "03a2efcf9d9a46f096a2e9434a1845f9"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "6c6e299453480fbfb2f2877b21dee43c"
server_key_iv: "61b11f6774c7c5f25cf0fa751b50bf8c"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.387542 27257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:10.402508 27257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:10.405349 27257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:10.405500 27257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:10.412022 27257 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:38299
I20260926 16:40:10.412324 27324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:38299 every 8 connection(s)
I20260926 16:40:10.413033 27257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:40:10.416016 27325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:40:10.416386 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 27257
I20260926 16:40:10.416543 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:40:10.416850 11898 external_mini_cluster.cc:1428] Setting key 464403be7962259598d8ad510bf4ce16
I20260926 16:40:10.422753 27325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Bootstrap starting.
I20260926 16:40:10.425567 27325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Neither blocks nor log segments found. Creating new log.
Sep 26 16:40:10 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:40:10.426505 27325 log.cc:824] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:10.428933 27325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: No bootstrap required, opened a new log
I20260926 16:40:10.431442 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:10.418091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.11.158.190:38299)
0926 16:40:10.418470 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:10.418495 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:10.418529 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:40:10.419200 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:10.419217 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:10.419522 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:10.419786 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:10.420690 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.421651 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.422811 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.423295 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.423407 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:10.426298 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:10.426303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:10.426317 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:10.426343 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:10.428313 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:10.428777 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:10.428783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:10.428788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:10.428919 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:10.429225 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:10.429229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:10.429230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:10.429735 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:10.429929 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:10.430146 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 16:40:10.430440 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":132,"threads_started":1}
I20260926 16:40:10.433419 27325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:10.433694 27325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:40:10.433758 27325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a2efcf9d9a46f096a2e9434a1845f9, State: Initialized, Role: FOLLOWER
I20260926 16:40:10.434161 27325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:10.434304 27325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:40:10.434367 27325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:40:10.434490 27325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:40:10.435381 27325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:10.435926 27325 leader_election.cc:302] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03a2efcf9d9a46f096a2e9434a1845f9; no voters:
I20260926 16:40:10.436417 27325 leader_election.cc:288] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:40:10.436515 27330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:40:10.436756 27330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 LEADER]: Becoming Leader. State: Replica: 03a2efcf9d9a46f096a2e9434a1845f9, State: Running, Role: LEADER
I20260926 16:40:10.437175 27330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:10.437824 27325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:40:10.438369 27332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03a2efcf9d9a46f096a2e9434a1845f9. Latest consensus state: current_term: 1 leader_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } } }
I20260926 16:40:10.438561 27332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: This master's current role is: LEADER
I20260926 16:40:10.439286 27331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } } }
I20260926 16:40:10.439388 27331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: This master's current role is: LEADER
I20260926 16:40:10.440115 27337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:40:10.443189 27337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:40:10.448509 27337 catalog_manager.cc:1347] Generated new cluster ID: e4d6c3cd654947219baf1df687f8abaa
I20260926 16:40:10.448576 27337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:40:10.465250 27337 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:40:10.466583 27337 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:40:10.478709 27337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Generated new TSK 0
I20260926 16:40:10.479449 27337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:40:10.515902 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38299
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:40:10.628660 27351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:10.629033 27351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:10.629123 27351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:10.633337 27351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:10.633392 27351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:10.633452 27351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:40:10.639114 27351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:10.640483 27351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:10.641860 27351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:10.651541 27356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:10.651535 27359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:10.651531 27357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:10.651877 27351 server_base.cc:1034] running on GCE node
I20260926 16:40:10.652320 27351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:10.653021 27351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:10.654217 27351 hybrid_clock.cc:648] HybridClock initialized: now 1790440810654186 us; error 44 us; skew 500 ppm
Sep 26 16:40:10 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440810, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:10.657300 27351 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:40:10.658180 27351 webserver.cc:460] Webserver started at http://127.11.158.129:33879/ using document root <none> and password file <none>
I20260926 16:40:10.658710 27351 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:10.658769 27351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:10.658934 27351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:40:10.661024 27351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "b536b651a30e4d6180ac028c31e5a9c4"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "a486c7c74f41413804f89b425f4215e8"
server_key_iv: "b4a0d2ee20460cb48fed7d445ac1ddb1"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.661547 27351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "b536b651a30e4d6180ac028c31e5a9c4"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "a486c7c74f41413804f89b425f4215e8"
server_key_iv: "b4a0d2ee20460cb48fed7d445ac1ddb1"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.665468 27351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:40:10.667980 27366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:10.669473 27351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:40:10.669615 27351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "b536b651a30e4d6180ac028c31e5a9c4"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "a486c7c74f41413804f89b425f4215e8"
server_key_iv: "b4a0d2ee20460cb48fed7d445ac1ddb1"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.669735 27351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:10.707137 27351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:10.710693 27351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:10.710913 27351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:10.711539 27351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:10.712440 27351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:40:10.712494 27351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:10.712561 27351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:40:10.712602 27351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:10.722060 27351 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:42167
I20260926 16:40:10.722083 27479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:42167 every 8 connection(s)
I20260926 16:40:10.723054 27351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
Sep 26 16:40:10 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440810, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:40:10.732906 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 27351
I20260926 16:40:10.733047 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:40:10.733327 11898 external_mini_cluster.cc:1428] Setting key 8eaceded656b6b122ed2b16875683fc2
I20260926 16:40:10.737072 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:10.724946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:49717 (local address 127.11.158.190:38299)
0926 16:40:10.725123 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:10.725128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:10.725977 (+ 849us) server_negotiation.cc:407] Connection header received
0926 16:40:10.727267 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:10.727270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:10.727329 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:10.727437 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:10.728714 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.729255 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.730388 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.730630 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.730745 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:10.733308 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:10.733312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:10.733314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:10.733337 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:10.734879 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:10.735444 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:10.735448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:10.735449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:10.735524 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:10.735884 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:10.735888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:10.735890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:10.736076 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:10.736187 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:10.736771 (+ 584us) server_negotiation.cc:299] Negotiation successful
0926 16:40:10.736899 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:40:10.737975 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:10.725368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:49717)
0926 16:40:10.725825 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:10.725868 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:10.727046 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:10.727615 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:10.727621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:10.727981 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:10.728558 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:10.728575 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.729386 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.729389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:10.730253 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:10.730262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.730787 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.730791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:10.730960 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:10.731528 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:10.731544 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:10.733117 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:10.735008 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:10.735015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:10.735031 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:10.735319 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:10.735626 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:10.735630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:10.735632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:10.735783 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:10.736188 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:10.736195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:10.736529 (+ 334us) client_negotiation.cc:769] Sending connection context
0926 16:40:10.736731 (+ 202us) client_negotiation.cc:240] Negotiation successful
0926 16:40:10.737073 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":126,"threads_started":1}
I20260926 16:40:10.739420 27480 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38299
I20260926 16:40:10.739707 27480 heartbeater.cc:464] Registering TS with master...
I20260926 16:40:10.740402 27480 heartbeater.cc:511] Master 127.11.158.190:38299 requested a full tablet report, sending...
I20260926 16:40:10.742296 27289 ts_manager.cc:194] Registered new tserver with Master: b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129:42167)
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:40:10.743749 27289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:49717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:40:10.761462 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38299
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:40:10.874584 27485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:10.874876 27485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:10.874919 27485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:10.879181 27485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:10.879241 27485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:10.879345 27485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:40:10.884706 27485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:10.885768 27485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:10.886794 27485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:10.897347 27493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:10.897347 27490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:10.897354 27491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:10.897689 27485 server_base.cc:1034] running on GCE node
I20260926 16:40:10.898150 27485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:10.898800 27485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:10.900002 27485 hybrid_clock.cc:648] HybridClock initialized: now 1790440810899983 us; error 31 us; skew 500 ppm
Sep 26 16:40:10 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440810, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:10.902861 27485 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:40:10.903677 27485 webserver.cc:460] Webserver started at http://127.11.158.130:45861/ using document root <none> and password file <none>
I20260926 16:40:10.904206 27485 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:10.904265 27485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:10.904531 27485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:40:10.906519 27485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "2ea18abdbf5f499ab20750b61790536c"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "5f0bc6516523f6e3aa780f6b95bbd254"
server_key_iv: "0daf8e825b75c642a53f9355b047673c"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.907037 27485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "2ea18abdbf5f499ab20750b61790536c"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "5f0bc6516523f6e3aa780f6b95bbd254"
server_key_iv: "0daf8e825b75c642a53f9355b047673c"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.910883 27485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:40:10.913403 27500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:10.914837 27485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:40:10.914948 27485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "2ea18abdbf5f499ab20750b61790536c"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "5f0bc6516523f6e3aa780f6b95bbd254"
server_key_iv: "0daf8e825b75c642a53f9355b047673c"
server_key_version: "encryptionkey@0"
I20260926 16:40:10.915046 27485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:10.938225 27485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:10.941977 27485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:10.942150 27485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:10.942687 27485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:10.943563 27485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:40:10.943615 27485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:10.943688 27485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:40:10.943727 27485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:10.953225 27485 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:42773
I20260926 16:40:10.953244 27613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:42773 every 8 connection(s)
I20260926 16:40:10.954178 27485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:40:10.958456 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 27485
I20260926 16:40:10.958607 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:40:10.958899 11898 external_mini_cluster.cc:1428] Setting key 7521ec7b4f09dcc980522541bf91f87e
Sep 26 16:40:10 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440810, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
I20260926 16:40:10.970553 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:10.956022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:47795 (local address 127.11.158.190:38299)
0926 16:40:10.956189 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:10.956193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:10.957109 (+ 916us) server_negotiation.cc:407] Connection header received
0926 16:40:10.958446 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:10.958450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:10.958516 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:10.958632 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:10.960106 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.960911 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.962195 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.962430 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.962555 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:10.965544 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:10.965552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:10.965555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:10.965581 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:10.967580 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:10.968436 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:10.968439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:10.968440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:10.968589 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:10.969136 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:10.969140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:10.969141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:10.969355 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:10.969518 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:10.970228 (+ 710us) server_negotiation.cc:299] Negotiation successful
0926 16:40:10.970379 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:40:10.971532 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:10.956465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:47795)
0926 16:40:10.956945 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:10.956993 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:10.958208 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:10.958856 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:10.958864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:10.959290 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:10.959925 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:10.959945 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.961068 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.961071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:10.962042 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:10.962053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:10.962604 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:10.962608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:10.962792 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:10.963453 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:10.963471 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:10.965317 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:10.967850 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:10.967857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:10.967884 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:10.968219 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:10.968744 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:10.968747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:10.968748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:10.968933 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:10.969499 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:10.969506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:10.969930 (+ 424us) client_negotiation.cc:769] Sending connection context
0926 16:40:10.970178 (+ 248us) client_negotiation.cc:240] Negotiation successful
0926 16:40:10.970600 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":123,"threads_started":1}
I20260926 16:40:10.972887 27614 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38299
I20260926 16:40:10.973181 27614 heartbeater.cc:464] Registering TS with master...
I20260926 16:40:10.973749 27614 heartbeater.cc:511] Master 127.11.158.190:38299 requested a full tablet report, sending...
I20260926 16:40:10.974942 27289 ts_manager.cc:194] Registered new tserver with Master: 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773)
I20260926 16:40:10.975574 27289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:47795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:40:10.987923 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38299
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:40:11.104539 27619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:11.104825 27619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:11.104867 27619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:11.108945 27619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:11.109016 27619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:11.109078 27619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:40:11.114427 27619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:11.115492 27619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:11.116598 27619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:11.126695 27625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:11.126695 27624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:11.126725 27627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:11.127180 27619 server_base.cc:1034] running on GCE node
I20260926 16:40:11.127588 27619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:11.128278 27619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:11.129483 27619 hybrid_clock.cc:648] HybridClock initialized: now 1790440811129474 us; error 55 us; skew 500 ppm
Sep 26 16:40:11 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440811, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:11.132388 27619 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:40:11.133306 27619 webserver.cc:460] Webserver started at http://127.11.158.131:33545/ using document root <none> and password file <none>
I20260926 16:40:11.133886 27619 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:11.133991 27619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:11.134231 27619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:40:11.136286 27619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "3a43f45af42e4efbada9c86bd4746c12"
format_stamp: "Formatted at 2026-09-26 16:40:11 on dist-test-slave-p39x"
server_key: "a408ca7c3a7917813702670e312e17a6"
server_key_iv: "b1c4398223b6fbbf9144eb759dfd0160"
server_key_version: "encryptionkey@0"
I20260926 16:40:11.136921 27619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "3a43f45af42e4efbada9c86bd4746c12"
format_stamp: "Formatted at 2026-09-26 16:40:11 on dist-test-slave-p39x"
server_key: "a408ca7c3a7917813702670e312e17a6"
server_key_iv: "b1c4398223b6fbbf9144eb759dfd0160"
server_key_version: "encryptionkey@0"
I20260926 16:40:11.141021 27619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:40:11.143402 27634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:11.144905 27619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:40:11.145044 27619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "3a43f45af42e4efbada9c86bd4746c12"
format_stamp: "Formatted at 2026-09-26 16:40:11 on dist-test-slave-p39x"
server_key: "a408ca7c3a7917813702670e312e17a6"
server_key_iv: "b1c4398223b6fbbf9144eb759dfd0160"
server_key_version: "encryptionkey@0"
I20260926 16:40:11.145162 27619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:11.162511 27619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:11.165527 27619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:11.165741 27619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:11.166349 27619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:11.167200 27619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:40:11.167271 27619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:11.167361 27619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:40:11.167404 27619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:11.176609 27619 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:37677
I20260926 16:40:11.176640 27747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:37677 every 8 connection(s)
I20260926 16:40:11.177549 27619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:40:11.184944 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 27619
I20260926 16:40:11.185077 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:40:11.185423 11898 external_mini_cluster.cc:1428] Setting key 8e22e05610533dab1d284d241b043d8c
Sep 26 16:40:11 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440811, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:40:11.191453 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.179472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:50821 (local address 127.11.158.190:38299)
0926 16:40:11.179648 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.179654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.180530 (+ 876us) server_negotiation.cc:407] Connection header received
0926 16:40:11.181801 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.181804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.181868 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.181953 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:11.183171 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.183720 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.184974 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.185316 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.185445 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.188056 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:11.188060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:11.188062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:11.188086 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:11.189504 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:11.189991 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:11.189994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:11.189995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:11.190068 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:11.190352 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:11.190354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:11.190355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:11.190617 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:11.190712 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.191211 (+ 499us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.191314 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 16:40:11.192287 27750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.179819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:50821)
0926 16:40:11.180304 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:11.180414 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:11.181600 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:11.182153 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:11.182160 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:11.182515 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:11.183014 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.183030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.183855 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.183858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.184839 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.184849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.185562 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.185569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.185722 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.186418 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:11.186435 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:11.187893 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:11.189614 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:11.189620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:11.189631 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:11.189900 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:11.190144 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:11.190146 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:11.190147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:11.190275 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:11.190722 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:11.190727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:11.190983 (+ 256us) client_negotiation.cc:769] Sending connection context
0926 16:40:11.191142 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 16:40:11.191385 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":125,"threads_started":1}
I20260926 16:40:11.193465 27748 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38299
I20260926 16:40:11.193703 27748 heartbeater.cc:464] Registering TS with master...
I20260926 16:40:11.194185 27748 heartbeater.cc:511] Master 127.11.158.190:38299 requested a full tablet report, sending...
I20260926 16:40:11.195242 27289 ts_manager.cc:194] Registered new tserver with Master: 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
I20260926 16:40:11.195852 27289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:50821
I20260926 16:40:11.199247 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:40:11.208191 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.201181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.11.158.190:38299)
0926 16:40:11.201356 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.201360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.201446 (+ 86us) server_negotiation.cc:407] Connection header received
0926 16:40:11.201550 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.201552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.201588 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.201653 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:11.202313 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.202837 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.203633 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.203836 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.203909 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.205301 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:11.205304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:11.205306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:11.205328 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:11.206620 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:11.207030 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:11.207033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:11.207034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:11.207105 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:11.207333 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:11.207335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:11.207336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:11.207534 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:11.207649 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.207894 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.207992 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 16:40:11.211207 27289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:40:11.214285 27289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:40:11.229208 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.224418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.11.158.131:37677)
0926 16:40:11.225146 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.225155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.225172 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:40:11.225226 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.225229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.225344 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.225488 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:11.225664 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.226555 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.227920 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.228551 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.228685 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.228816 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.228960 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.229046 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":127,"threads_started":1}
I20260926 16:40:11.229190 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.223836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:37677 (local address 127.0.0.1:57042)
0926 16:40:11.224802 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:11.224815 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:11.224899 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:11.225470 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:11.225473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:11.225490 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:11.225570 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.225575 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.226708 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.226711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.227804 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.227811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.228676 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.228680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.228781 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.228816 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 16:40:11.228927 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 16:40:11.229045 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":101,"threads_started":1}
I20260926 16:40:11.229355 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.223914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.11.158.129:42167)
0926 16:40:11.224296 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.224315 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.224369 (+ 54us) server_negotiation.cc:407] Connection header received
0926 16:40:11.224458 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.224464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.224632 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.224783 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:11.225088 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.226159 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.227792 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.228575 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.228686 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.228815 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.229109 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.229201 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":116,"threads_started":1}
I20260926 16:40:11.229728 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.223178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:42167 (local address 127.0.0.1:41454)
0926 16:40:11.224016 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:11.224031 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:11.224117 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:11.224802 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:11.224806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:11.224826 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:11.224950 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.224957 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.226288 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.226292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.227667 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.227678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.228878 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.228881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.228962 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.228978 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 16:40:11.229065 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:40:11.229138 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":129,"thread_start_us":109,"threads_started":1}
I20260926 16:40:11.230646 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.223116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:42773 (local address 127.0.0.1:49464)
0926 16:40:11.223547 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:11.223592 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:11.223714 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:11.224531 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:11.224537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:11.224559 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:11.224696 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.224707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.226172 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.226175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.227536 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.227545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.230234 (+ 2689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.230238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.230328 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.230352 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:40:11.230422 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 16:40:11.230511 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":46,"thread_start_us":101,"threads_started":1}
I20260926 16:40:11.231092 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.223234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.11.158.130:42773)
0926 16:40:11.223635 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.223646 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.223671 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:40:11.223922 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.223929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.224155 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.224311 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:11.224829 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.226034 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.227657 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.228545 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.228686 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.228815 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.230853 (+ 2038us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.230954 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":91,"threads_started":1}
I20260926 16:40:11.232483 27682 tablet_service.cc:1467] Processing CreateTablet for tablet 38e80ac0bdac487595876b35dd886434 (DEFAULT_TABLE table=test-table [id=7cb79b59f5d14d1fb84d496f327085eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:40:11.232685 27414 tablet_service.cc:1467] Processing CreateTablet for tablet 38e80ac0bdac487595876b35dd886434 (DEFAULT_TABLE table=test-table [id=7cb79b59f5d14d1fb84d496f327085eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:40:11.233578 27414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e80ac0bdac487595876b35dd886434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:40:11.233613 27682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e80ac0bdac487595876b35dd886434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:40:11.233667 27548 tablet_service.cc:1467] Processing CreateTablet for tablet 38e80ac0bdac487595876b35dd886434 (DEFAULT_TABLE table=test-table [id=7cb79b59f5d14d1fb84d496f327085eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:40:11.234597 27548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e80ac0bdac487595876b35dd886434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:40:11.240540 27765 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap starting.
I20260926 16:40:11.241923 27766 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Bootstrap starting.
I20260926 16:40:11.242785 27765 tablet_bootstrap.cc:654] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Neither blocks nor log segments found. Creating new log.
I20260926 16:40:11.243535 27765 log.cc:824] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:11.244047 27766 tablet_bootstrap.cc:654] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Neither blocks nor log segments found. Creating new log.
I20260926 16:40:11.244695 27767 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Bootstrap starting.
I20260926 16:40:11.244905 27766 log.cc:824] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:11.245711 27765 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: No bootstrap required, opened a new log
I20260926 16:40:11.245909 27765 ts_tablet_manager.cc:1397] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:40:11.246743 27766 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: No bootstrap required, opened a new log
I20260926 16:40:11.246948 27766 ts_tablet_manager.cc:1397] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260926 16:40:11.247196 27767 tablet_bootstrap.cc:654] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Neither blocks nor log segments found. Creating new log.
I20260926 16:40:11.248098 27767 log.cc:824] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:11.250157 27767 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: No bootstrap required, opened a new log
I20260926 16:40:11.250332 27767 ts_tablet_manager.cc:1397] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:40:11.250326 27765 raft_consensus.cc:348] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.250543 27765 raft_consensus.cc:374] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:40:11.250579 27765 raft_consensus.cc:729] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b536b651a30e4d6180ac028c31e5a9c4, State: Initialized, Role: FOLLOWER
I20260926 16:40:11.250478 27766 raft_consensus.cc:348] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.250725 27766 raft_consensus.cc:374] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:40:11.250772 27766 raft_consensus.cc:729] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a43f45af42e4efbada9c86bd4746c12, State: Initialized, Role: FOLLOWER
I20260926 16:40:11.250982 27765 consensus_queue.cc:260] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.251152 27766 consensus_queue.cc:260] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.251576 27480 heartbeater.cc:503] Master 127.11.158.190:38299 was elected leader, sending a full tablet report...
I20260926 16:40:11.251959 27765 ts_tablet_manager.cc:1428] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:40:11.252041 27766 ts_tablet_manager.cc:1428] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 16:40:11.252436 27748 heartbeater.cc:503] Master 127.11.158.190:38299 was elected leader, sending a full tablet report...
I20260926 16:40:11.254191 27767 raft_consensus.cc:348] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.254496 27767 raft_consensus.cc:374] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:40:11.254549 27767 raft_consensus.cc:729] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea18abdbf5f499ab20750b61790536c, State: Initialized, Role: FOLLOWER
I20260926 16:40:11.255010 27767 consensus_queue.cc:260] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.255615 27614 heartbeater.cc:503] Master 127.11.158.190:38299 was elected leader, sending a full tablet report...
I20260926 16:40:11.255976 27767 ts_tablet_manager.cc:1428] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260926 16:40:11.428565 27749 tablet.cc:2369] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:40:11.455339 27615 tablet.cc:2369] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:40:11.474649 27481 tablet.cc:2369] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:40:11.492748 27772 raft_consensus.cc:482] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:11.493019 27772 raft_consensus.cc:504] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.494305 27772 leader_election.cc:288] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129:42167)
I20260926 16:40:11.499722 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.494671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:51563 (local address 127.11.158.130:42773)
0926 16:40:11.494802 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.494807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.495119 (+ 312us) server_negotiation.cc:407] Connection header received
0926 16:40:11.495205 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.495212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.495272 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.495346 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:11.495659 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.496474 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.498380 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.499001 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.499151 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.499195 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.499506 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.499587 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:40:11.499722 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.494834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:53065 (local address 127.11.158.129:42167)
0926 16:40:11.494995 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.495001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.495023 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:40:11.495093 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.495096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.495155 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.495241 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:11.495621 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.496479 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.498380 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.499009 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.499140 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.499195 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.499506 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.499587 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:40:11.499758 27750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.494587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:42167 (local address 127.11.158.131:53065)
0926 16:40:11.494792 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:11.494808 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:11.494912 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:11.495267 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:11.495269 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:11.495284 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:11.495465 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.495474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.496605 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.496608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.498222 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.498234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.499150 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.499154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.499320 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.499387 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 16:40:11.499468 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:40:11.499599 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":283}
I20260926 16:40:11.499765 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.494587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:42773 (local address 127.11.158.131:51563)
0926 16:40:11.495007 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:11.495018 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:11.495126 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:11.495365 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:11.495367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:11.495379 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:11.495531 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.495541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.496605 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.496609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.498222 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:11.498234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.499150 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.499154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:11.499317 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.499387 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 16:40:11.499468 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:40:11.499599 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":301,"thread_start_us":101,"threads_started":1}
I20260926 16:40:11.500568 27434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e80ac0bdac487595876b35dd886434" candidate_uuid: "3a43f45af42e4efbada9c86bd4746c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b536b651a30e4d6180ac028c31e5a9c4" is_pre_election: true
I20260926 16:40:11.500555 27568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e80ac0bdac487595876b35dd886434" candidate_uuid: "3a43f45af42e4efbada9c86bd4746c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea18abdbf5f499ab20750b61790536c" is_pre_election: true
I20260926 16:40:11.500875 27568 raft_consensus.cc:2454] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a43f45af42e4efbada9c86bd4746c12 in term 0.
I20260926 16:40:11.500880 27434 raft_consensus.cc:2454] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a43f45af42e4efbada9c86bd4746c12 in term 0.
I20260926 16:40:11.501426 27635 leader_election.cc:302] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea18abdbf5f499ab20750b61790536c, 3a43f45af42e4efbada9c86bd4746c12; no voters:
I20260926 16:40:11.501719 27772 raft_consensus.cc:2781] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:40:11.501806 27772 raft_consensus.cc:482] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:40:11.501849 27772 raft_consensus.cc:3037] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:40:11.502866 27772 raft_consensus.cc:504] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.503226 27772 leader_election.cc:288] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [CANDIDATE]: Term 1 election: Requested vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129:42167)
I20260926 16:40:11.503567 27568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e80ac0bdac487595876b35dd886434" candidate_uuid: "3a43f45af42e4efbada9c86bd4746c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea18abdbf5f499ab20750b61790536c"
I20260926 16:40:11.503628 27434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38e80ac0bdac487595876b35dd886434" candidate_uuid: "3a43f45af42e4efbada9c86bd4746c12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b536b651a30e4d6180ac028c31e5a9c4"
I20260926 16:40:11.503695 27568 raft_consensus.cc:3037] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:40:11.503782 27434 raft_consensus.cc:3037] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:40:11.504825 27568 raft_consensus.cc:2454] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a43f45af42e4efbada9c86bd4746c12 in term 1.
I20260926 16:40:11.505010 27434 raft_consensus.cc:2454] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a43f45af42e4efbada9c86bd4746c12 in term 1.
I20260926 16:40:11.505162 27635 leader_election.cc:302] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea18abdbf5f499ab20750b61790536c, 3a43f45af42e4efbada9c86bd4746c12; no voters:
I20260926 16:40:11.505353 27772 raft_consensus.cc:2781] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:40:11.505587 27772 raft_consensus.cc:686] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 LEADER]: Becoming Leader. State: Replica: 3a43f45af42e4efbada9c86bd4746c12, State: Running, Role: LEADER
I20260926 16:40:11.505946 27772 consensus_queue.cc:237] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:11.509184 27287 catalog_manager.cc:5495] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131). New cstate: current_term: 1 leader_uuid: "3a43f45af42e4efbada9c86bd4746c12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } health_report { overall_health: HEALTHY } } }
I20260926 16:40:11.526712 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:11.523300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.11.158.131:37677)
0926 16:40:11.523432 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:11.523437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:11.523478 (+ 41us) server_negotiation.cc:407] Connection header received
0926 16:40:11.523608 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:11.523610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:11.523664 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:11.523750 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:40:11.523956 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.524531 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.525397 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:11.525617 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:11.525698 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:11.525956 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:40:11.526361 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:40:11.526466 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:11.526495 (+ 29us) server_negotiation.cc:299] Negotiation successful
0926 16:40:11.526544 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:40:11.532721 27434 raft_consensus.cc:1261] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Refusing update from remote peer 3a43f45af42e4efbada9c86bd4746c12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:40:11.532722 27568 raft_consensus.cc:1261] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Refusing update from remote peer 3a43f45af42e4efbada9c86bd4746c12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:40:11.533479 27775 consensus_queue.cc:1035] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:40:11.533665 27772 consensus_queue.cc:1035] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:40:11.541721 27780 mvcc.cc:205] Tried to move back new op lower bound from 7333645564030611456 to 7333645563930849280. Current Snapshot: MvccSnapshot[applied={T|T < 7333645564030611456}]
I20260926 16:40:11.551152 27289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:40:11.551352 27289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:40:11.553942 27289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Sending DeleteTablet for 3 replicas of tablet 38e80ac0bdac487595876b35dd886434
I20260926 16:40:11.555040 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 27351
I20260926 16:40:11.554999 27548 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e80ac0bdac487595876b35dd886434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:40:11 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49464
I20260926 16:40:11.555085 27414 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e80ac0bdac487595876b35dd886434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:40:11 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:41454
I20260926 16:40:11.555553 27682 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e80ac0bdac487595876b35dd886434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:40:11 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:57042
I20260926 16:40:11.556648 27788 tablet_replica.cc:330] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: stopping tablet replica
I20260926 16:40:11.556978 27788 raft_consensus.cc:2229] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:40:11.557327 27788 raft_consensus.cc:2258] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:40:11.558727 27788 ts_tablet_manager.cc:1905] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:40:11.561179 27788 ts_tablet_manager.cc:1918] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:40:11.561299 27788 log.cc:1196] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/38e80ac0bdac487595876b35dd886434
I20260926 16:40:11.561611 27789 tablet_replica.cc:330] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: stopping tablet replica
I20260926 16:40:11.561646 27788 ts_tablet_manager.cc:1939] T 38e80ac0bdac487595876b35dd886434 P 2ea18abdbf5f499ab20750b61790536c: Deleting consensus metadata
I20260926 16:40:11.562049 27789 raft_consensus.cc:2229] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:40:11.562530 27789 raft_consensus.cc:2258] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 16:40:11.562652 27275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:42167 (error 108)
I20260926 16:40:11.563288 27273 catalog_manager.cc:4841] TS 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773): tablet 38e80ac0bdac487595876b35dd886434 (table test-table [id=7cb79b59f5d14d1fb84d496f327085eb]) successfully deleted
I20260926 16:40:11.563896 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 27485
I20260926 16:40:11.565003 27789 ts_tablet_manager.cc:1905] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:40:11.565382 27759 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:40:11.563442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:42167 (local address 127.0.0.1:54694)
0926 16:40:11.565170 (+ 1728us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:11.565260 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:42167: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 16:40:11.565572 27275 catalog_manager.cc:4584] TS b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129:42167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38e80ac0bdac487595876b35dd886434: Network error: Client connection negotiation failed: client connection to 127.11.158.129:42167: connect: Connection reset by peer (error 104)
I20260926 16:40:11.567371 27789 ts_tablet_manager.cc:1918] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:40:11.567476 27789 log.cc:1196] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/38e80ac0bdac487595876b35dd886434
I20260926 16:40:11.567812 27789 ts_tablet_manager.cc:1939] T 38e80ac0bdac487595876b35dd886434 P 3a43f45af42e4efbada9c86bd4746c12: Deleting consensus metadata
I20260926 16:40:11.569005 27273 catalog_manager.cc:4841] TS 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677): tablet 38e80ac0bdac487595876b35dd886434 (table test-table [id=7cb79b59f5d14d1fb84d496f327085eb]) successfully deleted
I20260926 16:40:11.571811 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 27619
I20260926 16:40:11.577839 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 27257
I20260926 16:40:31.584615 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38299
--webserver_interface=127.11.158.190
--webserver_port=38389
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:40:31.695116 27791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:31.695463 27791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:31.695583 27791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:31.699791 27791 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 16:40:31.699874 27791 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 16:40:31.699901 27791 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 16:40:31.699931 27791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:40:31.699990 27791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:31.700053 27791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:31.700093 27791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:40:31.700127 27791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:40:31.705648 27791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=38389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:31.706857 27791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:31.707949 27791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:31.721845 27797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:31.721845 27800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:31.722117 27798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:31.722306 27791 server_base.cc:1034] running on GCE node
I20260926 16:40:31.723178 27791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:31.724561 27791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:31.725800 27791 hybrid_clock.cc:648] HybridClock initialized: now 1790440831725768 us; error 45 us; skew 500 ppm
Sep 26 16:40:31 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440831, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:31.736747 27791 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:40:31.737895 27791 webserver.cc:460] Webserver started at http://127.11.158.190:38389/ using document root <none> and password file <none>
I20260926 16:40:31.738538 27791 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:31.738657 27791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:31.743068 27791 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:40:31.746058 27807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:31.747473 27791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:40:31.747648 27791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "03a2efcf9d9a46f096a2e9434a1845f9"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "6c6e299453480fbfb2f2877b21dee43c"
server_key_iv: "61b11f6774c7c5f25cf0fa751b50bf8c"
server_key_version: "encryptionkey@0"
I20260926 16:40:31.748553 27791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:31.763862 27791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:31.767273 27791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:31.767480 27791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:31.775307 27791 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:38299
I20260926 16:40:31.775735 27859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:38299 every 8 connection(s)
I20260926 16:40:31.776659 27791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:40:31.781452 27860 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:40:31.781867 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 27791
I20260926 16:40:31.782305 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:42167
--local_ip_for_outbound_sockets=127.11.158.129
--tserver_master_addrs=127.11.158.190:38299
--webserver_port=33879
--webserver_interface=127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:40:31.783456 27860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Bootstrap starting.
I20260926 16:40:31.801433 27860 log.cc:824] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:31.810273 27860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:40:31.810688 27860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Bootstrap complete.
I20260926 16:40:31.815116 27860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:31.815675 27860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a2efcf9d9a46f096a2e9434a1845f9, State: Initialized, Role: FOLLOWER
I20260926 16:40:31.816251 27860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:31.816488 27860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:40:31.816591 27860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:40:31.816715 27860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:40:31.818853 27860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:31.819245 27860 leader_election.cc:302] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03a2efcf9d9a46f096a2e9434a1845f9; no voters:
I20260926 16:40:31.819618 27860 leader_election.cc:288] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:40:31.819737 27864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:40:31.819955 27864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 LEADER]: Becoming Leader. State: Replica: 03a2efcf9d9a46f096a2e9434a1845f9, State: Running, Role: LEADER
I20260926 16:40:31.820273 27864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:31.821009 27860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:40:31.821346 27864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03a2efcf9d9a46f096a2e9434a1845f9. Latest consensus state: current_term: 2 leader_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } } }
I20260926 16:40:31.821476 27864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: This master's current role is: LEADER
I20260926 16:40:31.821879 27865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } } }
I20260926 16:40:31.822081 27865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: This master's current role is: LEADER
I20260926 16:40:31.821936 27868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:40:31.827605 27868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:40:31.828560 27868 catalog_manager.cc:1259] Loaded cluster ID: e4d6c3cd654947219baf1df687f8abaa
I20260926 16:40:31.828642 27868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:40:31.831382 27868 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:40:31.832147 27868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Loaded TSK: 0
I20260926 16:40:31.853914 27868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Generated new TSK 1
I20260926 16:40:31.854280 27868 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:40:31.913936 27862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:31.914199 27862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:31.914242 27862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:31.918403 27862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:31.918464 27862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:31.918550 27862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:40:31.923946 27862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:42167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=33879
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:31.925062 27862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:31.926100 27862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:31.935894 27888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:31.935971 27890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:31.935966 27887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:31.936318 27862 server_base.cc:1034] running on GCE node
I20260926 16:40:31.936759 27862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:31.937428 27862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:31.938589 27862 hybrid_clock.cc:648] HybridClock initialized: now 1790440831938570 us; error 40 us; skew 500 ppm
Sep 26 16:40:31 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440831, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:31.941524 27862 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:40:31.942440 27862 webserver.cc:460] Webserver started at http://127.11.158.129:33879/ using document root <none> and password file <none>
I20260926 16:40:31.943006 27862 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:31.943079 27862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:31.947044 27862 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:40:31.949355 27897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:31.950670 27862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:40:31.950810 27862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "b536b651a30e4d6180ac028c31e5a9c4"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "a486c7c74f41413804f89b425f4215e8"
server_key_iv: "b4a0d2ee20460cb48fed7d445ac1ddb1"
server_key_version: "encryptionkey@0"
I20260926 16:40:31.951385 27862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:31.976737 27862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:31.979866 27862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:31.980084 27862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:31.980784 27862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:31.981953 27904 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:40:31.986060 27862 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:40:31.986150 27862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260926 16:40:31.986232 27862 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:40:31.987910 27862 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:40:31.987982 27862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 16:40:31.988274 27904 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap starting.
I20260926 16:40:31.998440 27862 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:42167
I20260926 16:40:31.998515 28011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:42167 every 8 connection(s)
I20260926 16:40:31.999594 27862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:40:32.007499 27904 log.cc:824] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:32.009578 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 27862
I20260926 16:40:32.010076 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:42773
--local_ip_for_outbound_sockets=127.11.158.130
--tserver_master_addrs=127.11.158.190:38299
--webserver_port=45861
--webserver_interface=127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:40:32 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440831, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:40:32.016032 27904 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:40:32.016513 27904 tablet_bootstrap.cc:492] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap complete.
I20260926 16:40:32.017896 27904 ts_tablet_manager.cc:1397] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Time spent bootstrapping tablet: real 0.030s user 0.012s sys 0.016s
I20260926 16:40:32.018594 28015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.001823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:58553)
0926 16:40:32.002897 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.002940 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.004170 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.005435 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.005443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.005906 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:32.006384 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.006398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.007736 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.007739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.008692 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.008703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.009450 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.009454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.009651 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.010341 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:32.010363 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:32.012133 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:32.014778 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.014787 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.014805 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.015217 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.015657 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.015661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.015665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.015865 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.016744 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:32.016750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:32.017045 (+ 295us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.017245 (+ 200us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.017635 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"thread_start_us":163,"threads_started":1}
I20260926 16:40:32.019212 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.001836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:58553 (local address 127.11.158.190:38299)
0926 16:40:32.002246 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.002266 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.003102 (+ 836us) server_negotiation.cc:407] Connection header received
0926 16:40:32.004685 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.004697 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.005023 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.005376 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:32.006581 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.007569 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.008823 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.009295 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.009479 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.012395 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:32.012401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:32.012416 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:32.012451 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:32.014636 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.015360 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.015366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.015371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.015539 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.015985 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.015988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.015990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.016560 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:32.016690 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.017419 (+ 729us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.017779 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":127,"threads_started":1}
I20260926 16:40:32.020776 27904 raft_consensus.cc:348] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:32.021307 27904 raft_consensus.cc:729] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b536b651a30e4d6180ac028c31e5a9c4, State: Initialized, Role: FOLLOWER
I20260926 16:40:32.021817 27904 consensus_queue.cc:260] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } }
I20260926 16:40:32.022137 28012 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38299
I20260926 16:40:32.022373 28012 heartbeater.cc:464] Registering TS with master...
I20260926 16:40:32.022920 27904 ts_tablet_manager.cc:1428] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 16:40:32.022956 28012 heartbeater.cc:511] Master 127.11.158.190:38299 requested a full tablet report, sending...
I20260926 16:40:32.025856 27824 ts_manager.cc:194] Registered new tserver with Master: b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129:42167)
I20260926 16:40:32.029440 27824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:58553
I20260926 16:40:32.032289 28012 heartbeater.cc:503] Master 127.11.158.190:38299 was elected leader, sending a full tablet report...
Sep 26 16:40:32 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440831, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.129@KRBTEST.COM
I20260926 16:40:32.040001 28019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.028893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:42167 (local address 127.0.0.1:44682)
0926 16:40:32.029708 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.029755 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.029897 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.030416 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.030422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.030819 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:32.030927 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.030934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.031895 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.031899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.032724 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.032736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.033539 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.033543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.033648 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.034256 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:32.034268 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:32.035976 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:32.038221 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.038230 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.038234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.038496 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.038819 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.038821 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.038823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.038908 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.039448 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:32.039453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:32.039572 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.039705 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.039866 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":119,"threads_started":1}
I20260926 16:40:32.040421 28020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.029014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.11.158.129:42167)
0926 16:40:32.029430 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.029442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.029874 (+ 432us) server_negotiation.cc:407] Connection header received
0926 16:40:32.030045 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.030051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.030268 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.030398 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:32.031064 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.031775 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.032853 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.033383 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.033502 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.036132 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:32.036143 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:32.036156 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:32.036184 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:32.038100 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.038621 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.038627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.038630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.038724 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.039001 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.039004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.039006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.039310 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:32.039442 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.039814 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.039987 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":122,"threads_started":1}
I20260926 16:40:32.041400 27946 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e80ac0bdac487595876b35dd886434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:40:11 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:44682
I20260926 16:40:32.042286 28021 tablet_replica.cc:330] stopping tablet replica
I20260926 16:40:32.042598 28021 raft_consensus.cc:2229] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:40:32.042855 28021 raft_consensus.cc:2258] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:40:32.043834 28021 ts_tablet_manager.cc:1905] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:40:32.047544 28021 ts_tablet_manager.cc:1918] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:40:32.047634 28021 log.cc:1196] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal/wals/38e80ac0bdac487595876b35dd886434
I20260926 16:40:32.047895 28021 ts_tablet_manager.cc:1939] T 38e80ac0bdac487595876b35dd886434 P b536b651a30e4d6180ac028c31e5a9c4: Deleting consensus metadata
I20260926 16:40:32.049973 27810 catalog_manager.cc:4841] TS b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129:42167): tablet 38e80ac0bdac487595876b35dd886434 (table test-table [id=7cb79b59f5d14d1fb84d496f327085eb]) successfully deleted
W20260926 16:40:32.136193 28017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:32.136566 28017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:32.136665 28017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:32.140892 28017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:32.141000 28017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:32.141093 28017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:40:32.147078 28017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:42773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=45861
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:32.148245 28017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:32.149336 28017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:32.159904 28029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:32.159904 28027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:32.159955 28026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:32.160028 28017 server_base.cc:1034] running on GCE node
I20260926 16:40:32.160705 28017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:32.161469 28017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:32.162664 28017 hybrid_clock.cc:648] HybridClock initialized: now 1790440832162646 us; error 28 us; skew 500 ppm
Sep 26 16:40:32 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440832, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:32.165822 28017 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:40:32.166893 28017 webserver.cc:460] Webserver started at http://127.11.158.130:45861/ using document root <none> and password file <none>
I20260926 16:40:32.167460 28017 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:32.167541 28017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:32.171672 28017 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:40:32.173990 28036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:32.175434 28017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:40:32.175606 28017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "2ea18abdbf5f499ab20750b61790536c"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "5f0bc6516523f6e3aa780f6b95bbd254"
server_key_iv: "0daf8e825b75c642a53f9355b047673c"
server_key_version: "encryptionkey@0"
I20260926 16:40:32.176239 28017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:32.187872 28017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:32.191236 28017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:32.191470 28017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:32.192107 28017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:32.193058 28017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:40:32.193132 28017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:32.193204 28017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:40:32.193255 28017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:32.202893 28017 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:42773
I20260926 16:40:32.202917 28149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:42773 every 8 connection(s)
I20260926 16:40:32.203926 28017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:40:32.207613 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28017
I20260926 16:40:32.208112 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:37677
--local_ip_for_outbound_sockets=127.11.158.131
--tserver_master_addrs=127.11.158.190:38299
--webserver_port=33545
--webserver_interface=127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:40:32 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440832, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:40:32.220696 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.205895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:49457 (local address 127.11.158.190:38299)
0926 16:40:32.206060 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.206065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.206985 (+ 920us) server_negotiation.cc:407] Connection header received
0926 16:40:32.208808 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.208811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.208863 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.208944 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:32.210443 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.211356 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.212715 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.213183 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.213330 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.216127 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:32.216135 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:32.216141 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:32.216182 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:32.218311 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.218925 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.218930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.218934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.219048 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.219402 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.219409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.219414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.219706 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:32.219819 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.220364 (+ 545us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.220517 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:40:32.221737 28152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.206351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:49457)
0926 16:40:32.206820 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.206863 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.208530 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.209178 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.209190 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.209764 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:32.210277 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.210291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.211519 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.211522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.212577 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.212589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.213395 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.213398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.213559 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.214320 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:32.214339 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:32.215905 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:32.218461 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.218467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.218482 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.218807 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.219154 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.219157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.219158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.219301 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.219818 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:32.219824 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:32.220083 (+ 259us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.220250 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.220718 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":127,"threads_started":1}
I20260926 16:40:32.223037 28150 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38299
I20260926 16:40:32.223318 28150 heartbeater.cc:464] Registering TS with master...
I20260926 16:40:32.223884 28150 heartbeater.cc:511] Master 127.11.158.190:38299 requested a full tablet report, sending...
I20260926 16:40:32.224994 27824 ts_manager.cc:194] Registered new tserver with Master: 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773)
I20260926 16:40:32.225594 27824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:49457
W20260926 16:40:32.327226 28153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:32.327541 28153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:32.327586 28153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:32.331987 28153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:32.332090 28153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:32.332188 28153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:40:32.337908 28153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:37677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=33545
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:32.339057 28153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:32.340114 28153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:32.349973 28161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:32.350055 28158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:32.349993 28159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:32.350220 28153 server_base.cc:1034] running on GCE node
I20260926 16:40:32.350695 28153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:32.351374 28153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:32.352591 28153 hybrid_clock.cc:648] HybridClock initialized: now 1790440832352552 us; error 51 us; skew 500 ppm
Sep 26 16:40:32 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440832, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:32.355645 28153 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:40:32.356709 28153 webserver.cc:460] Webserver started at http://127.11.158.131:33545/ using document root <none> and password file <none>
I20260926 16:40:32.357287 28153 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:32.357363 28153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:32.361426 28153 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:40:32.363616 28168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:32.364965 28153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:40:32.365088 28153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "3a43f45af42e4efbada9c86bd4746c12"
format_stamp: "Formatted at 2026-09-26 16:40:11 on dist-test-slave-p39x"
server_key: "a408ca7c3a7917813702670e312e17a6"
server_key_iv: "b1c4398223b6fbbf9144eb759dfd0160"
server_key_version: "encryptionkey@0"
I20260926 16:40:32.365666 28153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:32.386794 28153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:32.390267 28153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:32.390478 28153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:32.391105 28153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:32.391968 28153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:40:32.392040 28153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:32.392130 28153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:40:32.392174 28153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:32.401497 28153 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:37677
I20260926 16:40:32.401571 28281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:37677 every 8 connection(s)
I20260926 16:40:32.402570 28153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:40:32.405644 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28153
Sep 26 16:40:32 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440832, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:40:32.415886 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.406301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.11.158.190:38299)
0926 16:40:32.406528 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.406531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.406667 (+ 136us) server_negotiation.cc:407] Connection header received
0926 16:40:32.406798 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.406800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.406844 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.406923 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:32.407873 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.408516 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.409455 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.409671 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.409747 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.411343 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:32.411346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:32.411348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:32.411393 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:32.413549 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.414335 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.414341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.414343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.414464 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.414854 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.414860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.414862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.415123 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:32.415351 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.415523 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.415677 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":61,"threads_started":1}
I20260926 16:40:32.417793 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.404503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:50285 (local address 127.11.158.190:38299)
0926 16:40:32.404671 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.404676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.405562 (+ 886us) server_negotiation.cc:407] Connection header received
0926 16:40:32.407267 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.407269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.407299 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.407348 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:32.408792 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.409264 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.410795 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.411021 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.411105 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.413750 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:32.413753 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:32.413755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:32.413777 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:32.415530 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.416160 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.416164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.416165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.416253 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.416697 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.416700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.416702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.416914 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:32.417027 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.417545 (+ 518us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.417654 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:40:32.418615 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.404930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:50285)
0926 16:40:32.405413 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.405456 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.407041 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.407568 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.407576 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.408020 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:32.408638 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.408661 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.409399 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.409404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.410648 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.410662 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.411184 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.411188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.411409 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.411914 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:32.411930 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:32.413563 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:32.415654 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.415661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.415677 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.415977 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.416370 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:32.416374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:32.416376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:32.416582 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:32.417032 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:32.417040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:32.417295 (+ 255us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.417480 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.417809 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260926 16:40:32.419739 28282 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38299
I20260926 16:40:32.419987 28282 heartbeater.cc:464] Registering TS with master...
I20260926 16:40:32.420528 28282 heartbeater.cc:511] Master 127.11.158.190:38299 requested a full tablet report, sending...
I20260926 16:40:32.421572 27824 ts_manager.cc:194] Registered new tserver with Master: 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
I20260926 16:40:32.422184 27824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:50285
I20260926 16:40:32.429610 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:40:32.433662 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.430209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.11.158.190:38299)
0926 16:40:32.430336 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.430340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.430516 (+ 176us) server_negotiation.cc:407] Connection header received
0926 16:40:32.430660 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.430662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.430768 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.430863 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:40:32.431153 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.431664 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.432659 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.432868 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.432948 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.433219 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:40:32.433353 (+ 134us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 16:40:32.433558 (+ 205us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41502: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 16:40:32.433723 28014 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41502: token expired
W20260926 16:40:32.434270 27751 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.158.190:38299)
I20260926 16:40:32.441413 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.434691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.11.158.190:38299)
0926 16:40:32.434803 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.434806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.434820 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:40:32.434919 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.434921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.434954 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.435017 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:32.435584 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.436042 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.436866 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.437069 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.437148 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.438385 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:32.438388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:32.438390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:32.438412 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:32.439894 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.440276 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.440279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.440280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.440412 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:32.440653 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:32.440655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:32.440656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:32.440818 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:32.440917 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.441117 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.441205 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:40:32.443898 27824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:40:32.447309 27824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:40:32.455778 27946 tablet_service.cc:1467] Processing CreateTablet for tablet b00d7eabbd894bdcb8a7da0a1eb98b62 (DEFAULT_TABLE table=test-table [id=1bdaa1aa60af46f68a51d1d7722c3c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:40:32.456507 27946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00d7eabbd894bdcb8a7da0a1eb98b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:40:32.459434 27904 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap starting.
I20260926 16:40:32.460443 27904 tablet_bootstrap.cc:654] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Neither blocks nor log segments found. Creating new log.
I20260926 16:40:32.460492 28019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.454500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:42773 (local address 127.0.0.1:34746)
0926 16:40:32.454649 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.454666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.454772 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.456002 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.456005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.456025 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:32.456189 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.456197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.457541 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.457544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.458792 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.458803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.459907 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.459910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.460023 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.460074 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.460164 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.460252 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":73,"spinlock_wait_cycles":4992}
I20260926 16:40:32.460717 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.454727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.11.158.130:42773)
0926 16:40:32.455466 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.455479 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.455513 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:40:32.455593 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.455599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.455825 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.456183 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:32.456393 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.457427 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.458936 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.459774 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.459899 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.460001 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.460214 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.460311 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":169,"threads_started":1}
I20260926 16:40:32.461619 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.454776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:37677 (local address 127.0.0.1:38900)
0926 16:40:32.455099 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.455113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.455198 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.456045 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.456048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.456081 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:32.456391 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.456398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.457350 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.457354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.458804 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.458811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.461200 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.461204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.461289 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.461309 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.461401 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.461475 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":83,"spinlock_wait_cycles":384,"thread_start_us":111,"threads_started":1}
I20260926 16:40:32.461864 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.455242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.11.158.131:37677)
0926 16:40:32.455585 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.455597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.455619 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:40:32.455692 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.455696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.455907 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.456082 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:32.456487 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.457229 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.460055 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.461068 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.461212 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.461334 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.461607 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.461710 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":168,"threads_started":1}
I20260926 16:40:32.462203 27904 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: No bootstrap required, opened a new log
I20260926 16:40:32.462301 27904 ts_tablet_manager.cc:1397] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 16:40:32.462800 27904 raft_consensus.cc:348] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.462975 27904 raft_consensus.cc:374] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:40:32.463054 27904 raft_consensus.cc:729] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b536b651a30e4d6180ac028c31e5a9c4, State: Initialized, Role: FOLLOWER
I20260926 16:40:32.463204 27904 consensus_queue.cc:260] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.463482 27904 ts_tablet_manager.cc:1428] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:40:32.464061 28084 tablet_service.cc:1467] Processing CreateTablet for tablet b00d7eabbd894bdcb8a7da0a1eb98b62 (DEFAULT_TABLE table=test-table [id=1bdaa1aa60af46f68a51d1d7722c3c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:40:32.465242 28084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00d7eabbd894bdcb8a7da0a1eb98b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:40:32.465090 28216 tablet_service.cc:1467] Processing CreateTablet for tablet b00d7eabbd894bdcb8a7da0a1eb98b62 (DEFAULT_TABLE table=test-table [id=1bdaa1aa60af46f68a51d1d7722c3c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:40:32.466110 28216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00d7eabbd894bdcb8a7da0a1eb98b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:40:32.474246 28296 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Bootstrap starting.
I20260926 16:40:32.474651 28295 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Bootstrap starting.
I20260926 16:40:32.477474 28296 tablet_bootstrap.cc:654] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Neither blocks nor log segments found. Creating new log.
I20260926 16:40:32.477474 28295 tablet_bootstrap.cc:654] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Neither blocks nor log segments found. Creating new log.
I20260926 16:40:32.478379 28296 log.cc:824] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:32.478379 28295 log.cc:824] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:32.480434 28295 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: No bootstrap required, opened a new log
I20260926 16:40:32.480443 28296 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: No bootstrap required, opened a new log
I20260926 16:40:32.480643 28295 ts_tablet_manager.cc:1397] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:40:32.480645 28296 ts_tablet_manager.cc:1397] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:40:32.483481 28295 raft_consensus.cc:348] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.483481 28296 raft_consensus.cc:348] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.483706 28295 raft_consensus.cc:374] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:40:32.483740 28296 raft_consensus.cc:374] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:40:32.483739 28295 raft_consensus.cc:729] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a43f45af42e4efbada9c86bd4746c12, State: Initialized, Role: FOLLOWER
I20260926 16:40:32.483812 28296 raft_consensus.cc:729] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea18abdbf5f499ab20750b61790536c, State: Initialized, Role: FOLLOWER
I20260926 16:40:32.484225 28295 consensus_queue.cc:260] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.484234 28296 consensus_queue.cc:260] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.484880 28282 heartbeater.cc:503] Master 127.11.158.190:38299 was elected leader, sending a full tablet report...
I20260926 16:40:32.484880 28150 heartbeater.cc:503] Master 127.11.158.190:38299 was elected leader, sending a full tablet report...
I20260926 16:40:32.485249 28296 ts_tablet_manager.cc:1428] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:40:32.485248 28295 ts_tablet_manager.cc:1428] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260926 16:40:32.501077 28013 tablet.cc:2369] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:40:32.630931 28018 raft_consensus.cc:482] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:32.631170 28018 raft_consensus.cc:504] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.632361 28018 leader_election.cc:288] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
I20260926 16:40:32.636765 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.632824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:35269 (local address 127.11.158.130:42773)
0926 16:40:32.632972 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.632976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.633096 (+ 120us) server_negotiation.cc:407] Connection header received
0926 16:40:32.633307 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.633313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.633380 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.633425 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:32.633772 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.634601 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.635718 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.636146 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.636284 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.636341 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.636562 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.636641 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:40:32.636781 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.632719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:42773 (local address 127.11.158.129:35269)
0926 16:40:32.632976 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.632991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.633130 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.633552 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.633554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.633571 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:32.633663 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.633667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.634721 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.634723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.635574 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.635583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.636271 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.636275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.636389 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.636441 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.636544 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.636624 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":34,"thread_start_us":96,"threads_started":1}
I20260926 16:40:32.636875 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.633294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:51423 (local address 127.11.158.131:37677)
0926 16:40:32.633436 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.633444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.633488 (+ 44us) server_negotiation.cc:407] Connection header received
0926 16:40:32.633673 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.633677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.633758 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.633872 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:40:32.634005 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.634711 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.635799 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.636239 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.636409 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.636472 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.636654 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.636729 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:40:32.636896 28302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.633129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:37677 (local address 127.11.158.129:51423)
0926 16:40:32.633391 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.633407 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:32.633553 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:32.633871 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:32.633873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:32.633883 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:40:32.633935 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.633938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.634848 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.634852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.635657 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:32.635667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.636399 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.636403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:32.636505 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.636531 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:40:32.636638 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 16:40:32.636716 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260926 16:40:32.637686 28236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00d7eabbd894bdcb8a7da0a1eb98b62" candidate_uuid: "b536b651a30e4d6180ac028c31e5a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a43f45af42e4efbada9c86bd4746c12" is_pre_election: true
I20260926 16:40:32.638018 28236 raft_consensus.cc:2454] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b536b651a30e4d6180ac028c31e5a9c4 in term 0.
I20260926 16:40:32.638195 28104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00d7eabbd894bdcb8a7da0a1eb98b62" candidate_uuid: "b536b651a30e4d6180ac028c31e5a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea18abdbf5f499ab20750b61790536c" is_pre_election: true
I20260926 16:40:32.638552 28104 raft_consensus.cc:2454] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b536b651a30e4d6180ac028c31e5a9c4 in term 0.
I20260926 16:40:32.638540 27898 leader_election.cc:302] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a43f45af42e4efbada9c86bd4746c12, b536b651a30e4d6180ac028c31e5a9c4; no voters:
I20260926 16:40:32.638883 28018 raft_consensus.cc:2781] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:40:32.638990 28018 raft_consensus.cc:482] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:40:32.639060 28018 raft_consensus.cc:3037] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:40:32.640149 28018 raft_consensus.cc:504] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.640571 28018 leader_election.cc:288] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 1 election: Requested vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
I20260926 16:40:32.640949 28104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00d7eabbd894bdcb8a7da0a1eb98b62" candidate_uuid: "b536b651a30e4d6180ac028c31e5a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea18abdbf5f499ab20750b61790536c"
I20260926 16:40:32.641058 28236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00d7eabbd894bdcb8a7da0a1eb98b62" candidate_uuid: "b536b651a30e4d6180ac028c31e5a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a43f45af42e4efbada9c86bd4746c12"
I20260926 16:40:32.641093 28104 raft_consensus.cc:3037] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:40:32.641227 28236 raft_consensus.cc:3037] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:40:32.642238 28104 raft_consensus.cc:2454] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b536b651a30e4d6180ac028c31e5a9c4 in term 1.
I20260926 16:40:32.642302 28236 raft_consensus.cc:2454] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b536b651a30e4d6180ac028c31e5a9c4 in term 1.
I20260926 16:40:32.642611 27898 leader_election.cc:302] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea18abdbf5f499ab20750b61790536c, b536b651a30e4d6180ac028c31e5a9c4; no voters:
I20260926 16:40:32.642781 28018 raft_consensus.cc:2781] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:40:32.643015 28018 raft_consensus.cc:686] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 LEADER]: Becoming Leader. State: Replica: b536b651a30e4d6180ac028c31e5a9c4, State: Running, Role: LEADER
I20260926 16:40:32.643291 28018 consensus_queue.cc:237] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:32.646435 27823 catalog_manager.cc:5495] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129). New cstate: current_term: 1 leader_uuid: "b536b651a30e4d6180ac028c31e5a9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } health_report { overall_health: HEALTHY } } }
W20260926 16:40:32.653563 28283 tablet.cc:2369] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:40:32.699967 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.696226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.11.158.129:42167)
0926 16:40:32.696571 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:32.696579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:32.696593 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:40:32.696668 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:32.696670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:32.696725 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:32.696846 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:40:32.697129 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.697645 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.698468 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:32.698686 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:32.698765 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:32.699081 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:40:32.699600 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:40:32.699689 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:32.699789 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 16:40:32.699840 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":146,"threads_started":1}
I20260926 16:40:32.704002 28236 raft_consensus.cc:1261] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 FOLLOWER]: Refusing update from remote peer b536b651a30e4d6180ac028c31e5a9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:40:32.704015 28104 raft_consensus.cc:1261] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Refusing update from remote peer b536b651a30e4d6180ac028c31e5a9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:40:32.704751 28303 consensus_queue.cc:1035] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:40:32.704936 28018 consensus_queue.cc:1035] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:40:32.705183 28151 tablet.cc:2369] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:40:32.712498 28307 mvcc.cc:205] Tried to move back new op lower bound from 7333645650749145088 to 7333645650509721600. Current Snapshot: MvccSnapshot[applied={T|T < 7333645650749145088}]
I20260926 16:40:32.713392 28309 mvcc.cc:205] Tried to move back new op lower bound from 7333645650749145088 to 7333645650509721600. Current Snapshot: MvccSnapshot[applied={T|T < 7333645650749145088}]
I20260926 16:40:32.721285 27824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:40:32.721494 27824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:40:32.724164 27824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Sending DeleteTablet for 3 replicas of tablet b00d7eabbd894bdcb8a7da0a1eb98b62
I20260926 16:40:32.725071 27946 tablet_service.cc:1514] Processing DeleteTablet for tablet b00d7eabbd894bdcb8a7da0a1eb98b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:40:32 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:44682
I20260926 16:40:32.725291 28216 tablet_service.cc:1514] Processing DeleteTablet for tablet b00d7eabbd894bdcb8a7da0a1eb98b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:40:32 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:38900
I20260926 16:40:32.725595 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 27862
I20260926 16:40:32.726675 28084 tablet_service.cc:1514] Processing DeleteTablet for tablet b00d7eabbd894bdcb8a7da0a1eb98b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:40:32 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:34746
I20260926 16:40:32.727125 28319 tablet_replica.cc:330] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: stopping tablet replica
I20260926 16:40:32.727334 28318 tablet_replica.cc:330] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: stopping tablet replica
I20260926 16:40:32.727486 28319 raft_consensus.cc:2229] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:40:32.727677 28318 raft_consensus.cc:2229] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:40:32.727854 28319 raft_consensus.cc:2258] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:40:32.727970 28318 raft_consensus.cc:2258] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:40:32.729476 28319 ts_tablet_manager.cc:1905] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:40:32.729509 28318 ts_tablet_manager.cc:1905] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:40:32.731745 28319 ts_tablet_manager.cc:1918] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:40:32.731873 28319 log.cc:1196] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/b00d7eabbd894bdcb8a7da0a1eb98b62
I20260926 16:40:32.732187 28319 ts_tablet_manager.cc:1939] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 2ea18abdbf5f499ab20750b61790536c: Deleting consensus metadata
I20260926 16:40:32.733033 28318 ts_tablet_manager.cc:1918] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:40:32.733145 28318 log.cc:1196] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/b00d7eabbd894bdcb8a7da0a1eb98b62
I20260926 16:40:32.733490 28318 ts_tablet_manager.cc:1939] T b00d7eabbd894bdcb8a7da0a1eb98b62 P 3a43f45af42e4efbada9c86bd4746c12: Deleting consensus metadata
I20260926 16:40:32.733664 27808 catalog_manager.cc:4841] TS 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773): tablet b00d7eabbd894bdcb8a7da0a1eb98b62 (table test-table [id=1bdaa1aa60af46f68a51d1d7722c3c38]) successfully deleted
W20260926 16:40:32.734524 27810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:42167 (error 108)
I20260926 16:40:32.734750 27808 catalog_manager.cc:4841] TS 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677): tablet b00d7eabbd894bdcb8a7da0a1eb98b62 (table test-table [id=1bdaa1aa60af46f68a51d1d7722c3c38]) successfully deleted
I20260926 16:40:32.735133 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 28017
I20260926 16:40:32.736159 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:32.735295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:42167 (local address 127.0.0.1:44692)
0926 16:40:32.736021 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:32.736084 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":637}
W20260926 16:40:32.736316 27810 catalog_manager.cc:4584] TS b536b651a30e4d6180ac028c31e5a9c4 (127.11.158.129:42167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b00d7eabbd894bdcb8a7da0a1eb98b62: Network error: Client connection negotiation failed: client connection to 127.11.158.129:42167: connect: Connection refused (error 111)
I20260926 16:40:32.741740 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 28153
I20260926 16:40:32.747232 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 27791
I20260926 16:40:52.753683 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38299
--webserver_interface=127.11.158.190
--webserver_port=38389
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:40:52.866632 28321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:52.866911 28321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:52.866954 28321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:52.871065 28321 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 16:40:52.871129 28321 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 16:40:52.871145 28321 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 16:40:52.871158 28321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:40:52.871177 28321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:52.871194 28321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:52.871256 28321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:40:52.871285 28321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:40:52.876752 28321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=38389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:52.877986 28321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:52.879051 28321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:52.888608 28327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:52.888608 28326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:52.888656 28321 server_base.cc:1034] running on GCE node
W20260926 16:40:52.888612 28329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:52.889549 28321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:52.890791 28321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:52.891973 28321 hybrid_clock.cc:648] HybridClock initialized: now 1790440852891952 us; error 36 us; skew 500 ppm
Sep 26 16:40:52 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440852, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:52.895097 28321 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:40:52.896066 28321 webserver.cc:460] Webserver started at http://127.11.158.190:38389/ using document root <none> and password file <none>
I20260926 16:40:52.896705 28321 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:52.896770 28321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:52.901070 28321 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:40:52.903638 28336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:52.905333 28321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:40:52.905509 28321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "03a2efcf9d9a46f096a2e9434a1845f9"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "6c6e299453480fbfb2f2877b21dee43c"
server_key_iv: "61b11f6774c7c5f25cf0fa751b50bf8c"
server_key_version: "encryptionkey@0"
I20260926 16:40:52.906164 28321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:52.922566 28321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:52.926600 28321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:52.926837 28321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:52.934522 28321 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:38299
I20260926 16:40:52.934801 28388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:38299 every 8 connection(s)
I20260926 16:40:52.935766 28321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:40:52.940063 28389 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:40:52.940451 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28321
I20260926 16:40:52.940865 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:42167
--local_ip_for_outbound_sockets=127.11.158.129
--tserver_master_addrs=127.11.158.190:38299
--webserver_port=33879
--webserver_interface=127.11.158.129
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:40:52.941910 28389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Bootstrap starting.
I20260926 16:40:52.959556 28389 log.cc:824] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:52.970527 28389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:40:52.970934 28389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Bootstrap complete.
I20260926 16:40:52.975060 28389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:52.975481 28389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a2efcf9d9a46f096a2e9434a1845f9, State: Initialized, Role: FOLLOWER
I20260926 16:40:52.975973 28389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:52.976114 28389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:40:52.976195 28389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:40:52.976310 28389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 2 FOLLOWER]: Advancing to term 3
I20260926 16:40:52.978381 28389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:52.978722 28389 leader_election.cc:302] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03a2efcf9d9a46f096a2e9434a1845f9; no voters:
I20260926 16:40:52.979068 28389 leader_election.cc:288] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 16:40:52.979158 28393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 16:40:52.979378 28393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [term 3 LEADER]: Becoming Leader. State: Replica: 03a2efcf9d9a46f096a2e9434a1845f9, State: Running, Role: LEADER
I20260926 16:40:52.979756 28393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } }
I20260926 16:40:52.980409 28389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:40:52.981163 28394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03a2efcf9d9a46f096a2e9434a1845f9. Latest consensus state: current_term: 3 leader_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } } }
I20260926 16:40:52.981292 28394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: This master's current role is: LEADER
I20260926 16:40:52.982146 28395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a2efcf9d9a46f096a2e9434a1845f9" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38299 } } }
I20260926 16:40:52.982285 28395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9 [sys.catalog]: This master's current role is: LEADER
I20260926 16:40:52.982568 28400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:40:52.986765 28400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:40:52.987318 28400 catalog_manager.cc:1259] Loaded cluster ID: e4d6c3cd654947219baf1df687f8abaa
I20260926 16:40:52.987393 28400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:40:52.989272 28400 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:40:52.989890 28400 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Loaded TSK: 0
I20260926 16:40:52.989975 28400 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03a2efcf9d9a46f096a2e9434a1845f9: Loaded TSK: 1
F20260926 16:40:52.990584 28400 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbcdff5f9ad google::LogMessage::Fail() at ??:0
@ 0x7fbcdff6288a google::LogMessage::SendToLog() at ??:0
@ 0x7fbcdff5f4de google::LogMessage::Flush() at ??:0
@ 0x7fbcdff60a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbce2c2318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbce2c1bf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbcea8c47bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbcea89e99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbcea89f178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fbcea8e0df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fbcea8ff82b std::function<>::operator()() at ??:0
@ 0x7fbcea89eb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fbcea89ff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbcea89c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fbcea8e0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbce21b13d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbce21b1c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fbce21b33f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbce21a2512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbce37e66db start_thread at ??:0
@ 0x7fbcdea5d61f clone at ??:0
W20260926 16:40:53.095314 28391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:53.095604 28391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:53.095651 28391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:53.100032 28391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:53.100102 28391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:53.100201 28391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:40:53.105979 28391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:42167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=33879
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:53.107151 28391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:53.108227 28391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:53.118408 28437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:53.118549 28391 server_base.cc:1034] running on GCE node
W20260926 16:40:53.118408 28439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:53.118748 28436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:53.119200 28391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:53.119884 28391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:53.121064 28391 hybrid_clock.cc:648] HybridClock initialized: now 1790440853121036 us; error 50 us; skew 500 ppm
Sep 26 16:40:53 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:53.123955 28391 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:40:53.124922 28391 webserver.cc:460] Webserver started at http://127.11.158.129:33879/ using document root <none> and password file <none>
I20260926 16:40:53.125571 28391 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:53.125669 28391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:53.129837 28391 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:40:53.132264 28446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:53.133656 28391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:40:53.133793 28391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "b536b651a30e4d6180ac028c31e5a9c4"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "a486c7c74f41413804f89b425f4215e8"
server_key_iv: "b4a0d2ee20460cb48fed7d445ac1ddb1"
server_key_version: "encryptionkey@0"
I20260926 16:40:53.134359 28391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:53.164528 28391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:53.167973 28391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:53.168220 28391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:53.168932 28391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:53.170205 28453 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:40:53.175819 28391 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:40:53.175935 28391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260926 16:40:53.176023 28391 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:40:53.177932 28391 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:40:53.177996 28391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260926 16:40:53.178274 28453 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap starting.
I20260926 16:40:53.189688 28391 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:42167
I20260926 16:40:53.189742 28560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:42167 every 8 connection(s)
I20260926 16:40:53.191084 28391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:40:53.198401 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28391
I20260926 16:40:53.198904 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:42773
--local_ip_for_outbound_sockets=127.11.158.130
--tserver_master_addrs=127.11.158.190:38299
--webserver_port=45861
--webserver_interface=127.11.158.130
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:40:53.199343 28453 log.cc:824] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:40:53.207460 28453 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:40:53.207892 28453 tablet_bootstrap.cc:492] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Bootstrap complete.
I20260926 16:40:53.209326 28453 ts_tablet_manager.cc:1397] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Time spent bootstrapping tablet: real 0.031s user 0.026s sys 0.004s
W20260926 16:40:53.212087 28563 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:40:53.193547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:38923)
0926 16:40:53.194202 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.194266 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:53.195312 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:53.210970 (+ 15658us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":169,"threads_started":1}
W20260926 16:40:53.212769 28561 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:38299 (0 consecutive failures): Network error: Failed to ping master at 127.11.158.190:38299: Client connection negotiation failed: client connection to 127.11.158.190:38299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 16:40:53.212445 28453 raft_consensus.cc:348] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:53.212972 28453 raft_consensus.cc:729] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b536b651a30e4d6180ac028c31e5a9c4, State: Initialized, Role: FOLLOWER
I20260926 16:40:53.213500 28453 consensus_queue.cc:260] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:53.213662 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.213385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:54243)
0926 16:40:53.213525 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.213595 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 16:40:53.214345 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.214096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:37711)
0926 16:40:53.214234 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.214278 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260926 16:40:53.214537 28561 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:40:53.214566 28453 ts_tablet_manager.cc:1428] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:40:53.215032 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.214825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:44103)
0926 16:40:53.214926 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.214977 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 16:40:53.215602 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.215401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:34383)
0926 16:40:53.215496 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.215548 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 16:40:53.315335 28565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:53.315593 28565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:53.315635 28565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:53.319679 28565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:53.319743 28565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:53.319842 28565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:40:53.325162 28565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:42773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=45861
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:53.326336 28565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:53.327373 28565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:53.337132 28574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:53.337139 28572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:53.337139 28571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:53.337203 28565 server_base.cc:1034] running on GCE node
I20260926 16:40:53.337909 28565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:53.338634 28565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:53.339843 28565 hybrid_clock.cc:648] HybridClock initialized: now 1790440853339826 us; error 29 us; skew 500 ppm
Sep 26 16:40:53 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:53.342748 28565 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:40:53.343676 28565 webserver.cc:460] Webserver started at http://127.11.158.130:45861/ using document root <none> and password file <none>
I20260926 16:40:53.344259 28565 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:53.344370 28565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:53.348227 28565 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260926 16:40:53.350389 28581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:53.351644 28565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:40:53.351771 28565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "2ea18abdbf5f499ab20750b61790536c"
format_stamp: "Formatted at 2026-09-26 16:40:10 on dist-test-slave-p39x"
server_key: "5f0bc6516523f6e3aa780f6b95bbd254"
server_key_iv: "0daf8e825b75c642a53f9355b047673c"
server_key_version: "encryptionkey@0"
I20260926 16:40:53.352324 28565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:53.364678 28565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:53.367728 28565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:53.367947 28565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:53.368631 28565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:53.369508 28565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:40:53.369555 28565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:53.369596 28565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:40:53.369640 28565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:53.378937 28565 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:42773
I20260926 16:40:53.378945 28694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:42773 every 8 connection(s)
I20260926 16:40:53.379997 28565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:40:53.383960 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.382199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:37775)
0926 16:40:53.382656 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.382766 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":130,"threads_started":1}
W20260926 16:40:53.384480 28695 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:38299 (0 consecutive failures): Network error: Failed to ping master at 127.11.158.190:38299: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
I20260926 16:40:53.385146 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.384900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:60511)
0926 16:40:53.385034 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.385085 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 16:40:53.385825 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.385643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:56247)
0926 16:40:53.385736 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.385777 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 16:40:53.386045 28695 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:40:53.389179 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28565
I20260926 16:40:53.389602 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:37677
--local_ip_for_outbound_sockets=127.11.158.131
--tserver_master_addrs=127.11.158.190:38299
--webserver_port=33545
--webserver_interface=127.11.158.131
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:40:53.502871 28698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:40:53.503185 28698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:40:53.503258 28698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:40:53.507397 28698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:40:53.507495 28698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:40:53.507592 28698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:40:53.513343 28698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:37677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=33545
--tserver_master_addrs=127.11.158.190:38299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:40:53.514524 28698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:40:53.515645 28698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:40:53.525607 28703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:53.525631 28704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:40:53.525607 28706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:40:53.525898 28698 server_base.cc:1034] running on GCE node
I20260926 16:40:53.526487 28698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:40:53.527227 28698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:40:53.528442 28698 hybrid_clock.cc:648] HybridClock initialized: now 1790440853528416 us; error 34 us; skew 500 ppm
Sep 26 16:40:53 dist-test-slave-p39x krb5kdc[27244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:40:53.531322 28698 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:40:53.532294 28698 webserver.cc:460] Webserver started at http://127.11.158.131:33545/ using document root <none> and password file <none>
I20260926 16:40:53.532904 28698 fs_manager.cc:359] Metadata directory not provided
I20260926 16:40:53.532994 28698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:40:53.536934 28698 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:40:53.539029 28713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:53.540236 28698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:40:53.540388 28698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "3a43f45af42e4efbada9c86bd4746c12"
format_stamp: "Formatted at 2026-09-26 16:40:11 on dist-test-slave-p39x"
server_key: "a408ca7c3a7917813702670e312e17a6"
server_key_iv: "b1c4398223b6fbbf9144eb759dfd0160"
server_key_version: "encryptionkey@0"
I20260926 16:40:53.540936 28698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:40:53.557013 28698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:40:53.560284 28698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:40:53.560515 28698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:40:53.561137 28698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:40:53.561986 28698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:40:53.562055 28698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:53.562124 28698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:40:53.562165 28698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:40:53.570982 28698 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:37677
I20260926 16:40:53.571008 28826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:37677 every 8 connection(s)
I20260926 16:40:53.571982 28698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:40:53.575974 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.574145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:33513)
0926 16:40:53.574621 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.574729 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":117,"threads_started":1}
I20260926 16:40:53.576102 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28698
W20260926 16:40:53.576572 28827 heartbeater.cc:644] Failed to heartbeat to 127.11.158.190:38299 (0 consecutive failures): Network error: Failed to ping master at 127.11.158.190:38299: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
I20260926 16:40:53.577226 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.576980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:45671)
0926 16:40:53.577101 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.577148 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 16:40:53.577798 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:53.577601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:57959)
0926 16:40:53.577701 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:53.577743 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260926 16:40:53.577960 28827 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:40:54.216890 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:54.216446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:51521)
0926 16:40:54.216699 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:54.216765 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260926 16:40:54.387024 28832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:54.386608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:49117)
0926 16:40:54.386852 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:54.386907 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260926 16:40:54.579047 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:54.578613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:33615)
0926 16:40:54.578884 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:54.578946 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":74,"threads_started":1}
I20260926 16:40:54.798916 28834 raft_consensus.cc:482] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:54.799122 28834 raft_consensus.cc:504] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:54.800279 28834 leader_election.cc:288] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
Sep 26 16:40:54 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.130@KRBTEST.COM
Sep 26 16:40:54 dist-test-slave-p39x krb5kdc[27244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440853, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.131@KRBTEST.COM
I20260926 16:40:54.818195 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:54.800790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:34723 (local address 127.11.158.130:42773)
0926 16:40:54.801250 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:54.801279 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:54.801325 (+ 46us) server_negotiation.cc:407] Connection header received
0926 16:40:54.802256 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:54.802266 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:54.802711 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:54.803065 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:54.804860 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.806448 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.808617 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.809332 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.809519 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:54.813082 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:54.813088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:54.813103 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:54.813135 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:54.815312 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:54.815834 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:54.815839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:54.815843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:54.815962 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:54.816322 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:54.816328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:54.816353 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:54.816963 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:54.817105 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:54.817690 (+ 585us) server_negotiation.cc:299] Negotiation successful
0926 16:40:54.817996 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":74,"threads_started":1}
I20260926 16:40:54.818190 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:54.801291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:51357 (local address 127.11.158.131:37677)
0926 16:40:54.801707 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0926 16:40:54.801726 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:40:54.801764 (+ 38us) server_negotiation.cc:407] Connection header received
0926 16:40:54.802535 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:40:54.802543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:40:54.802850 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:40:54.803092 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:40:54.804860 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.806448 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.808556 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.809332 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.809519 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:54.813376 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:40:54.813383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:40:54.813400 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:40:54.813440 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:40:54.815312 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:54.816388 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:54.816397 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:54.816405 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:54.816540 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:40:54.816805 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:40:54.816808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:40:54.816810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:40:54.817282 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:40:54.817379 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:40:54.817720 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 16:40:54.818004 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":115,"threads_started":1}
I20260926 16:40:54.818171 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:54.801027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:37677 (local address 127.11.158.129:51357)
0926 16:40:54.801296 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:54.801308 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:54.801397 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:54.803132 (+ 1735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:54.803139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:54.803731 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:54.804655 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:54.804679 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.806636 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.806640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:54.808427 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:54.808441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.809745 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.809750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:54.809926 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:54.810788 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:54.810810 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:54.813159 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:54.815812 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:54.815813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:54.815815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:54.815972 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:54.816632 (+ 660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:54.816634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:54.816636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:54.816710 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:54.817388 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:54.817392 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:54.817470 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 16:40:54.817588 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 16:40:54.817931 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":376,"spinlock_wait_cycles":44544,"thread_start_us":60,"threads_started":1}
I20260926 16:40:54.818172 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:54.800615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:42773 (local address 127.11.158.129:34723)
0926 16:40:54.800948 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:54.800961 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:40:54.801105 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:40:54.803132 (+ 2027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:40:54.803139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:40:54.803731 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:40:54.804655 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:54.804679 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.806663 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.806668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:54.808430 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:40:54.808446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:40:54.809500 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:40:54.809506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:40:54.809890 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:40:54.810795 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:40:54.810808 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:40:54.812793 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:40:54.815441 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:54.815446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:54.815458 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:54.815722 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:54.816067 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:40:54.816070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:40:54.816072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:40:54.816217 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:40:54.817081 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:40:54.817086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:40:54.817308 (+ 222us) client_negotiation.cc:769] Sending connection context
0926 16:40:54.817529 (+ 221us) client_negotiation.cc:240] Negotiation successful
0926 16:40:54.817985 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":491,"spinlock_wait_cycles":24832,"thread_start_us":100,"threads_started":1}
W20260926 16:40:54.821188 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
W20260926 16:40:54.821422 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
I20260926 16:40:54.821486 28447 leader_election.cc:302] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b536b651a30e4d6180ac028c31e5a9c4; no voters: 2ea18abdbf5f499ab20750b61790536c, 3a43f45af42e4efbada9c86bd4746c12
I20260926 16:40:54.821817 28834 raft_consensus.cc:2726] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:40:55.218088 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:55.217789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:48039)
0926 16:40:55.217937 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:55.218002 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 16:40:55.388227 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:55.387813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:39541)
0926 16:40:55.388071 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:55.388131 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
I20260926 16:40:55.580304 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:55.579871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:41911)
0926 16:40:55.580145 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:55.580211 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260926 16:40:56.219393 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:56.218909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:34095)
0926 16:40:56.219232 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:56.219292 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":110,"threads_started":1}
I20260926 16:40:56.389798 28842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:56.389272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:36825)
0926 16:40:56.389609 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:56.389679 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":84,"threads_started":1}
I20260926 16:40:56.518934 28843 raft_consensus.cc:482] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:56.519070 28843 raft_consensus.cc:504] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:56.519423 28843 leader_election.cc:288] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
W20260926 16:40:56.520582 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
W20260926 16:40:56.520757 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
I20260926 16:40:56.520829 28447 leader_election.cc:302] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b536b651a30e4d6180ac028c31e5a9c4; no voters: 2ea18abdbf5f499ab20750b61790536c, 3a43f45af42e4efbada9c86bd4746c12
I20260926 16:40:56.521062 28843 raft_consensus.cc:2726] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:40:56.581696 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:56.581237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:35237)
0926 16:40:56.581524 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:56.581588 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260926 16:40:57.220829 28845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:57.220309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:38581)
0926 16:40:57.220661 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:57.220728 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":98,"threads_started":1}
I20260926 16:40:57.391160 28846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:57.390715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:55167)
0926 16:40:57.391001 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:57.391065 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":78,"threads_started":1}
I20260926 16:40:57.583055 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:57.582569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:60449)
0926 16:40:57.582891 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:57.582958 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
I20260926 16:40:58.222082 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:58.221694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:53495)
0926 16:40:58.221937 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:58.221990 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260926 16:40:58.392807 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:58.392103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:54371)
0926 16:40:58.392472 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:58.392702 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
I20260926 16:40:58.563378 28850 raft_consensus.cc:482] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:40:58.563520 28850 raft_consensus.cc:504] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:40:58.564002 28850 leader_election.cc:288] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
W20260926 16:40:58.565018 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
W20260926 16:40:58.565172 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
I20260926 16:40:58.565215 28447 leader_election.cc:302] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b536b651a30e4d6180ac028c31e5a9c4; no voters: 2ea18abdbf5f499ab20750b61790536c, 3a43f45af42e4efbada9c86bd4746c12
I20260926 16:40:58.565368 28850 raft_consensus.cc:2726] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:40:58.584311 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:58.583890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:41523)
0926 16:40:58.584165 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:58.584218 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":92,"threads_started":1}
I20260926 16:40:59.223451 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:59.222962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:47519)
0926 16:40:59.223261 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:59.223325 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":80,"threads_started":1}
I20260926 16:40:59.394263 28853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:59.393771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:45271)
0926 16:40:59.394073 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:59.394138 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":119,"threads_started":1}
I20260926 16:40:59.585526 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:40:59.585072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:56385)
0926 16:40:59.585345 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 16:40:59.585430 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260926 16:41:00.224974 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:00.224532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:54125)
0926 16:41:00.224803 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:00.224872 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260926 16:41:00.395697 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:00.395182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:48549)
0926 16:41:00.395517 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:00.395596 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":84,"threads_started":1}
I20260926 16:41:00.586885 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:00.586396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:58169)
0926 16:41:00.586732 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:00.586791 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":85,"threads_started":1}
I20260926 16:41:01.226153 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:01.225715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:52861)
0926 16:41:01.225982 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:01.226055 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
I20260926 16:41:01.397078 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:01.396647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:57349)
0926 16:41:01.396915 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:01.396978 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260926 16:41:01.588419 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:01.587825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:48373)
0926 16:41:01.588176 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:01.588248 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":121,"threads_started":1}
I20260926 16:41:02.227326 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:02.226922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:53487)
0926 16:41:02.227177 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:02.227233 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":92,"threads_started":1}
I20260926 16:41:02.398414 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:02.397927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:51239)
0926 16:41:02.398241 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:02.398308 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":99,"threads_started":1}
I20260926 16:41:02.589882 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:02.589356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:51681)
0926 16:41:02.589716 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:02.589785 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260926 16:41:03.193118 28864 raft_consensus.cc:482] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:41:03.193262 28864 raft_consensus.cc:504] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ea18abdbf5f499ab20750b61790536c" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 42773 } } peers { permanent_uuid: "3a43f45af42e4efbada9c86bd4746c12" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 37677 } } peers { permanent_uuid: "b536b651a30e4d6180ac028c31e5a9c4" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42167 } }
I20260926 16:41:03.193679 28864 leader_election.cc:288] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773), 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677)
W20260926 16:41:03.194550 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ea18abdbf5f499ab20750b61790536c (127.11.158.130:42773): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
W20260926 16:41:03.194713 28447 leader_election.cc:341] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a43f45af42e4efbada9c86bd4746c12 (127.11.158.131:37677): Not found: Tablet not found: b00d7eabbd894bdcb8a7da0a1eb98b62
I20260926 16:41:03.194787 28447 leader_election.cc:302] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b536b651a30e4d6180ac028c31e5a9c4; no voters: 2ea18abdbf5f499ab20750b61790536c, 3a43f45af42e4efbada9c86bd4746c12
I20260926 16:41:03.194979 28864 raft_consensus.cc:2726] T b00d7eabbd894bdcb8a7da0a1eb98b62 P b536b651a30e4d6180ac028c31e5a9c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:41:03.228911 28865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:03.228424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:60771)
0926 16:41:03.228731 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:03.228809 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":130,"threads_started":1}
I20260926 16:41:03.399809 28866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:03.399319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:57375)
0926 16:41:03.399652 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:03.399713 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":107,"threads_started":1}
I20260926 16:41:03.591317 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:03.590804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:38201)
0926 16:41:03.591123 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:03.591187 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
I20260926 16:41:04.230335 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:04.229894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:41567)
0926 16:41:04.230177 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:04.230234 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":116,"threads_started":1}
I20260926 16:41:04.401160 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:04.400688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:37741)
0926 16:41:04.400997 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:04.401061 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":103,"threads_started":1}
I20260926 16:41:04.592801 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:04.592277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:60415)
0926 16:41:04.592631 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:04.592697 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":128,"threads_started":1}
I20260926 16:41:05.231535 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:05.231120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:60409)
0926 16:41:05.231376 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:05.231436 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
I20260926 16:41:05.402320 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:05.401926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:42755)
0926 16:41:05.402163 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:05.402220 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260926 16:41:05.594172 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:05.593743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:55277)
0926 16:41:05.594011 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:05.594076 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":77,"threads_started":1}
I20260926 16:41:06.233016 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:06.232528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:44077)
0926 16:41:06.232849 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:06.232917 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":97,"threads_started":1}
I20260926 16:41:06.403738 28875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:06.403202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:58163)
0926 16:41:06.403507 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:06.403581 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":92,"threads_started":1}
I20260926 16:41:06.595495 28876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:06.595080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:60721)
0926 16:41:06.595332 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:06.595394 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260926 16:41:07.234368 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:07.233922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:51469)
0926 16:41:07.234218 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:07.234271 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":121,"threads_started":1}
I20260926 16:41:07.405071 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:07.404634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:48123)
0926 16:41:07.404912 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:07.404975 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260926 16:41:07.597028 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:07.596548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.131:53463)
0926 16:41:07.596856 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:07.596922 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":81,"threads_started":1}
I20260926 16:41:08.235929 28880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:08.235416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.129:53759)
0926 16:41:08.235749 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:08.235812 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":95,"threads_started":1}
I20260926 16:41:08.406472 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:08.406079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38299 (local address 127.11.158.130:40641)
0926 16:41:08.406320 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:08.406376 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.190:38299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260926 16:41:08.582942 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 28391
I20260926 16:41:08.589015 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 28565
I20260926 16:41:08.595284 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 28698
2026-09-26T16:41:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59541 ms)
[----------] 30 tests from SecurityITest (245652 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:08 dist-test-slave-p39x krb5kdc[28884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:08 dist-test-slave-p39x krb5kdc[28884](info): set up 2 sockets
Sep 26 16:41:08 dist-test-slave-p39x krb5kdc[28884](info): commencing operation
krb5kdc: starting...
W20260926 16:41:09.648957 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:09 dist-test-slave-p39x krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:09.780531 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:45771
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:45771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:09.892673 28897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:09.893007 28897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:09.893090 28897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:09.897380 28897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:09.897485 28897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:09.897523 28897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:09.897591 28897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:09.897631 28897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:09.903192 28897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:45771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:09.904371 28897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:09.905457 28897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:09.917308 28902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:09.917548 28905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:09.917322 28897 server_base.cc:1034] running on GCE node
W20260926 16:41:09.917444 28903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:09.918262 28897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:09.919461 28897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:09.920650 28897 hybrid_clock.cc:648] HybridClock initialized: now 1790440869920638 us; error 27 us; skew 500 ppm
Sep 26 16:41:09 dist-test-slave-p39x krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440869, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:09.927663 28897 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:09.928767 28897 webserver.cc:460] Webserver started at http://127.11.158.190:36019/ using document root <none> and password file <none>
I20260926 16:41:09.929363 28897 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:09.929468 28897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:09.929714 28897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:09.931797 28897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "810d1ee6fedb46bd9895a239752c1b1c"
format_stamp: "Formatted at 2026-09-26 16:41:09 on dist-test-slave-p39x"
server_key: "360c4a64e07a582e914418bc80c51595"
server_key_iv: "f5e0c6dbe8bfb9a5d6d42e2b0ac796d6"
server_key_version: "encryptionkey@0"
I20260926 16:41:09.932473 28897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "810d1ee6fedb46bd9895a239752c1b1c"
format_stamp: "Formatted at 2026-09-26 16:41:09 on dist-test-slave-p39x"
server_key: "360c4a64e07a582e914418bc80c51595"
server_key_iv: "f5e0c6dbe8bfb9a5d6d42e2b0ac796d6"
server_key_version: "encryptionkey@0"
I20260926 16:41:09.936131 28897 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:09.938637 28912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:09.939834 28897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:41:09.939987 28897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "810d1ee6fedb46bd9895a239752c1b1c"
format_stamp: "Formatted at 2026-09-26 16:41:09 on dist-test-slave-p39x"
server_key: "360c4a64e07a582e914418bc80c51595"
server_key_iv: "f5e0c6dbe8bfb9a5d6d42e2b0ac796d6"
server_key_version: "encryptionkey@0"
I20260926 16:41:09.940112 28897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:09.953702 28897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:09.956754 28897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:09.956977 28897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:09.964274 28897 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:45771
I20260926 16:41:09.964288 28964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:45771 every 8 connection(s)
I20260926 16:41:09.965313 28897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:09.967064 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28897
I20260926 16:41:09.967226 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:09.967535 11898 external_mini_cluster.cc:1428] Setting key 1c26604eca507204bb6e3296aaef3fbf
I20260926 16:41:09.968411 28965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:09.975438 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c: Bootstrap starting.
I20260926 16:41:09.978206 28965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c: Neither blocks nor log segments found. Creating new log.
Sep 26 16:41:09 dist-test-slave-p39x krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:09.979184 28965 log.cc:824] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:09.981534 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c: No bootstrap required, opened a new log
I20260926 16:41:09.984175 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:09.969021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.11.158.190:45771)
0926 16:41:09.969449 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:09.969469 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:09.969522 (+ 53us) server_negotiation.cc:407] Connection header received
0926 16:41:09.970389 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:09.970413 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:09.970896 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:09.971313 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:09.972961 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:09.974550 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:09.975497 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:09.976176 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:09.976353 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:09.979212 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:09.979218 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:09.979232 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:09.979293 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:09.981115 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:09.981679 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:09.981684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:09.981688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:09.981804 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:09.982128 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:09.982131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:09.982132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:09.982586 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:09.982744 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:09.983054 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 16:41:09.983404 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260926 16:41:09.985630 28965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d1ee6fedb46bd9895a239752c1b1c" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45771 } }
I20260926 16:41:09.985883 28965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:09.985967 28965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 810d1ee6fedb46bd9895a239752c1b1c, State: Initialized, Role: FOLLOWER
I20260926 16:41:09.986444 28965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d1ee6fedb46bd9895a239752c1b1c" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45771 } }
I20260926 16:41:09.986603 28965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:09.986688 28965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:09.986788 28965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:09.987757 28965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d1ee6fedb46bd9895a239752c1b1c" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45771 } }
I20260926 16:41:09.988119 28965 leader_election.cc:302] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 810d1ee6fedb46bd9895a239752c1b1c; no voters:
I20260926 16:41:09.988489 28965 leader_election.cc:288] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:09.988621 28970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:09.988906 28970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [term 1 LEADER]: Becoming Leader. State: Replica: 810d1ee6fedb46bd9895a239752c1b1c, State: Running, Role: LEADER
I20260926 16:41:09.989300 28970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d1ee6fedb46bd9895a239752c1b1c" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45771 } }
I20260926 16:41:09.989751 28965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:09.991041 28971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "810d1ee6fedb46bd9895a239752c1b1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d1ee6fedb46bd9895a239752c1b1c" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45771 } } }
I20260926 16:41:09.991094 28972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 810d1ee6fedb46bd9895a239752c1b1c. Latest consensus state: current_term: 1 leader_uuid: "810d1ee6fedb46bd9895a239752c1b1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810d1ee6fedb46bd9895a239752c1b1c" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45771 } } }
I20260926 16:41:09.991163 28971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:09.991194 28972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:09.991535 28979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:09.995273 28979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:10.001139 28979 catalog_manager.cc:1347] Generated new cluster ID: 044a19b5cd8748339c093f8fa0aa1b6c
I20260926 16:41:10.001225 28979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:10.021535 28979 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:10.022905 28979 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:10.029595 28979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c: Generated new TSK 0
I20260926 16:41:10.030457 28979 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:41:10.067494 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:45771
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:10.189495 28991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:10.189788 28991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:10.189836 28991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:10.194419 28991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:10.194492 28991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:10.194597 28991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:41:10.200312 28991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:10.201591 28991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:10.202864 28991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:10.213111 28996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:10.213061 28999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:10.213163 28991 server_base.cc:1034] running on GCE node
W20260926 16:41:10.213097 28997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:10.213786 28991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:10.214442 28991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:10.215646 28991 hybrid_clock.cc:648] HybridClock initialized: now 1790440870215626 us; error 29 us; skew 500 ppm
Sep 26 16:41:10 dist-test-slave-p39x krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:10.218459 28991 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:41:10.219384 28991 webserver.cc:460] Webserver started at http://127.11.158.129:45709/ using document root <none> and password file <none>
I20260926 16:41:10.219977 28991 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:10.220078 28991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:10.220396 28991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:10.222503 28991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "6a06faa1eb934adda28e3ee327ddf3bc"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "61c9265bec9fdc6f37f32772fba5baeb"
server_key_iv: "f2786471516f99080273b448f24a42a5"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.223110 28991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "6a06faa1eb934adda28e3ee327ddf3bc"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "61c9265bec9fdc6f37f32772fba5baeb"
server_key_iv: "f2786471516f99080273b448f24a42a5"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.226930 28991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:41:10.229390 29006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.230636 28991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:10.230790 28991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "6a06faa1eb934adda28e3ee327ddf3bc"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "61c9265bec9fdc6f37f32772fba5baeb"
server_key_iv: "f2786471516f99080273b448f24a42a5"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.230923 28991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:10.247418 28991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:10.250869 28991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:10.251113 28991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:10.251768 28991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:10.252771 28991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:10.252856 28991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.252934 28991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:10.253005 28991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.262840 28991 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:46335
I20260926 16:41:10.262856 29119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:46335 every 8 connection(s)
I20260926 16:41:10.263936 28991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:41:10.265064 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 28991
I20260926 16:41:10.265190 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:41:10.265486 11898 external_mini_cluster.cc:1428] Setting key 4be30c71c6b5f6451dd90d58d18f90c1
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
Sep 26 16:41:10 dist-test-slave-p39x krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:10.283696 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.266164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:44057 (local address 127.11.158.190:45771)
0926 16:41:10.266346 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:10.266351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:10.267436 (+ 1085us) server_negotiation.cc:407] Connection header received
0926 16:41:10.269210 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:10.269213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:10.269264 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:10.269370 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:10.270862 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.271556 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.273480 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.273730 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.273817 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.279498 (+ 5681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:10.279503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:10.279505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:10.279532 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:10.281379 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.281933 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.281937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.281961 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.282041 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.282420 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.282424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.282426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.282661 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:10.282792 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:10.283340 (+ 548us) server_negotiation.cc:299] Negotiation successful
0926 16:41:10.283499 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 16:41:10.284684 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.266660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:45771 (local address 127.11.158.129:44057)
0926 16:41:10.267248 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:10.267310 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:10.269007 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:10.269549 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:10.269556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:10.270017 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:10.270701 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.270723 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.271698 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.271703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.273349 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.273361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.273893 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.273899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.274127 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.274776 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:10.274814 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:10.279286 (+ 4472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:10.281499 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:10.281505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:10.281518 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:10.281807 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:10.282157 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:10.282161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:10.282163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:10.282307 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:10.282790 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:10.282797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:10.283076 (+ 279us) client_negotiation.cc:769] Sending connection context
0926 16:41:10.283279 (+ 203us) client_negotiation.cc:240] Negotiation successful
0926 16:41:10.283642 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":175,"threads_started":1}
I20260926 16:41:10.286003 29120 heartbeater.cc:347] Connected to a master server at 127.11.158.190:45771
I20260926 16:41:10.286274 29120 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:10.286867 29120 heartbeater.cc:511] Master 127.11.158.190:45771 requested a full tablet report, sending...
I20260926 16:41:10.288489 28929 ts_manager.cc:194] Registered new tserver with Master: 6a06faa1eb934adda28e3ee327ddf3bc (127.11.158.129:46335)
I20260926 16:41:10.289942 28929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:44057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:41:10.296306 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:45771
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:10.406026 29125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:10.406286 29125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:10.406329 29125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:10.410247 29125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:10.410298 29125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:10.410356 29125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:41:10.415994 29125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:10.417026 29125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:10.417989 29125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:10.428203 29125 server_base.cc:1034] running on GCE node
W20260926 16:41:10.428143 29131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:10.428143 29130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:10.428143 29133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:10.428992 29125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:10.429627 29125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:10.430807 29125 hybrid_clock.cc:648] HybridClock initialized: now 1790440870430784 us; error 39 us; skew 500 ppm
Sep 26 16:41:10 dist-test-slave-p39x krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:10.433885 29125 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:41:10.434844 29125 webserver.cc:460] Webserver started at http://127.11.158.130:33007/ using document root <none> and password file <none>
I20260926 16:41:10.435453 29125 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:10.435523 29125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:10.435747 29125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:10.437922 29125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "345a7ad6d4514e209408c7817fb82652"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "aefd951cff1d8547fbb8622b33490ca1"
server_key_iv: "a26eb0625e10b545d81ebce3ca54a967"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.438531 29125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "345a7ad6d4514e209408c7817fb82652"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "aefd951cff1d8547fbb8622b33490ca1"
server_key_iv: "a26eb0625e10b545d81ebce3ca54a967"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.442462 29125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:41:10.445221 29140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.446419 29125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:10.446581 29125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "345a7ad6d4514e209408c7817fb82652"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "aefd951cff1d8547fbb8622b33490ca1"
server_key_iv: "a26eb0625e10b545d81ebce3ca54a967"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.446708 29125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:10.465795 29125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:10.469239 29125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:10.469458 29125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:10.470063 29125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:10.470899 29125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:10.470975 29125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.471051 29125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:10.471107 29125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.479892 29125 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:39107
I20260926 16:41:10.479914 29253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:39107 every 8 connection(s)
I20260926 16:41:10.480849 29125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:41:10.482833 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 29125
I20260926 16:41:10.482975 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:41:10.483309 11898 external_mini_cluster.cc:1428] Setting key 84d7bf36d537af6dd19248011963268b
Sep 26 16:41:10 dist-test-slave-p39x krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:41:10.496754 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.482670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:37387 (local address 127.11.158.190:45771)
0926 16:41:10.482845 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:10.482851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:10.483859 (+ 1008us) server_negotiation.cc:407] Connection header received
0926 16:41:10.485489 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:10.485492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:10.485533 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:10.485604 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:10.487045 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.487896 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.489176 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.489602 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.489761 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.492747 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:10.492750 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:10.492752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:10.492775 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:10.494372 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.494868 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.494871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.494873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.494950 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.495472 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.495475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.495477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.495686 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:10.495778 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:10.496513 (+ 735us) server_negotiation.cc:299] Negotiation successful
0926 16:41:10.496627 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:41:10.497982 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.483135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:45771 (local address 127.11.158.130:37387)
0926 16:41:10.483692 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:10.483750 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:10.485279 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:10.485787 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:10.485799 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:10.486303 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:10.486902 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.486915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.488021 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.488023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.489052 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.489061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.489781 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.489787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.489934 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.490558 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:10.490582 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:10.492565 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:10.494482 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:10.494488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:10.494501 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:10.494758 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:10.495039 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:10.495041 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:10.495042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:10.495252 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:10.495865 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:10.495876 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:10.496219 (+ 343us) client_negotiation.cc:769] Sending connection context
0926 16:41:10.496442 (+ 223us) client_negotiation.cc:240] Negotiation successful
0926 16:41:10.496826 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":146,"threads_started":1}
I20260926 16:41:10.499183 29254 heartbeater.cc:347] Connected to a master server at 127.11.158.190:45771
I20260926 16:41:10.499449 29254 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:10.499958 29254 heartbeater.cc:511] Master 127.11.158.190:45771 requested a full tablet report, sending...
I20260926 16:41:10.501101 28929 ts_manager.cc:194] Registered new tserver with Master: 345a7ad6d4514e209408c7817fb82652 (127.11.158.130:39107)
I20260926 16:41:10.501689 28929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:37387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:41:10.513330 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:45771
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:10.625862 29259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:10.626134 29259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:10.626178 29259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:10.630345 29259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:10.630414 29259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:10.630476 29259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:41:10.636102 29259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:45771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:10.637202 29259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:10.638155 29259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:10.648198 29265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:10.648303 29259 server_base.cc:1034] running on GCE node
W20260926 16:41:10.648190 29267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:10.648185 29264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:10.649037 29259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:10.649819 29259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:10.651041 29259 hybrid_clock.cc:648] HybridClock initialized: now 1790440870651005 us; error 51 us; skew 500 ppm
Sep 26 16:41:10 dist-test-slave-p39x krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:10.654201 29259 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:41:10.655157 29259 webserver.cc:460] Webserver started at http://127.11.158.131:44039/ using document root <none> and password file <none>
I20260926 16:41:10.655776 29259 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:10.655875 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:10.656124 29259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:10.658329 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "eb08123776de4522a0eaf1573747f698"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "ea3d5b65d200bbc742445344ac46e7ff"
server_key_iv: "af1835b90da63cd100c3312759111da9"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.658941 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "eb08123776de4522a0eaf1573747f698"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "ea3d5b65d200bbc742445344ac46e7ff"
server_key_iv: "af1835b90da63cd100c3312759111da9"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.662997 29259 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:41:10.665701 29274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.667133 29259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:41:10.667305 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "eb08123776de4522a0eaf1573747f698"
format_stamp: "Formatted at 2026-09-26 16:41:10 on dist-test-slave-p39x"
server_key: "ea3d5b65d200bbc742445344ac46e7ff"
server_key_iv: "af1835b90da63cd100c3312759111da9"
server_key_version: "encryptionkey@0"
I20260926 16:41:10.667440 29259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:10.686304 29259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:10.689857 29259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:10.690032 29259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:10.690748 29259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:10.691694 29259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:10.691742 29259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.691782 29259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:10.691798 29259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:10.701371 29259 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:46281
I20260926 16:41:10.701490 29387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:46281 every 8 connection(s)
I20260926 16:41:10.702351 29259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:41:10.710289 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 29259
I20260926 16:41:10.710462 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:41:10.710801 11898 external_mini_cluster.cc:1428] Setting key c017714ff82a91ed686e796e866ccdd5
Sep 26 16:41:10 dist-test-slave-p39x krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440870, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:10.716565 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.704268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:40451 (local address 127.11.158.190:45771)
0926 16:41:10.704473 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:10.704477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:10.705445 (+ 968us) server_negotiation.cc:407] Connection header received
0926 16:41:10.706731 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:10.706734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:10.706786 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:10.706866 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:10.708046 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.708660 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.709770 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.709983 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.710063 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.713000 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:10.713004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:10.713012 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:10.713037 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:10.714416 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.714989 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.714993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.714995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.715076 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.715425 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.715430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.715432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.715631 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:10.715742 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:10.716243 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 16:41:10.716393 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260926 16:41:10.717376 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.704783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:45771 (local address 127.11.158.131:40451)
0926 16:41:10.705270 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:10.705314 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:10.706506 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:10.707050 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:10.707058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:10.707410 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:10.707898 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.707912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.708770 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.708773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.709663 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.709672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.710140 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.710146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.710365 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.711106 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:10.711131 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:10.712820 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:10.714593 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:10.714599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:10.714614 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:10.714879 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:10.715173 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:10.715178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:10.715180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:10.715328 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:10.715740 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:10.715748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:10.716008 (+ 260us) client_negotiation.cc:769] Sending connection context
0926 16:41:10.716204 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 16:41:10.716568 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":128,"threads_started":1}
I20260926 16:41:10.718550 29388 heartbeater.cc:347] Connected to a master server at 127.11.158.190:45771
I20260926 16:41:10.718786 29388 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:10.719293 29388 heartbeater.cc:511] Master 127.11.158.190:45771 requested a full tablet report, sending...
I20260926 16:41:10.720369 28929 ts_manager.cc:194] Registered new tserver with Master: eb08123776de4522a0eaf1573747f698 (127.11.158.131:46281)
I20260926 16:41:10.720979 28929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:40451
I20260926 16:41:10.725796 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:41:10.735621 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.727990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.11.158.190:45771)
0926 16:41:10.728198 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:10.728201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:10.728288 (+ 87us) server_negotiation.cc:407] Connection header received
0926 16:41:10.728464 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:10.728466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:10.728516 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:10.728585 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:10.729283 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.729811 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.730702 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.730939 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.731028 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.732587 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:10.732591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:10.732593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:10.732617 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:10.733865 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.734247 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.734249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.734251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.734322 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:10.734605 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:10.734608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:10.734609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:10.734891 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:10.735003 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:10.735290 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 16:41:10.735407 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260926 16:41:10.738921 28929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:41:10.742022 28929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:41:10.756855 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.750858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:46281 (local address 127.0.0.1:43856)
0926 16:41:10.751589 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:10.751640 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:10.751769 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:10.752637 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:10.752640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:10.752671 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:10.752863 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.752871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.754363 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.754367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.755354 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.755363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.756265 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.756268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.756406 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.756466 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 16:41:10.756567 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 16:41:10.756687 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":3,"spinlock_wait_cycles":12288,"thread_start_us":127,"threads_started":1}
I20260926 16:41:10.757017 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.750767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:46335 (local address 127.0.0.1:57554)
0926 16:41:10.751590 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:10.751640 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:10.751769 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:10.752632 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:10.752638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:10.752668 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:10.752864 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.752871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.754060 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.754064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.755413 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.755423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.756543 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.756547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.756635 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.756659 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:41:10.756741 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 16:41:10.756806 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":63,"spinlock_wait_cycles":7040,"thread_start_us":86,"threads_started":1}
I20260926 16:41:10.757431 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.750896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57554 (local address 127.11.158.129:46335)
0926 16:41:10.752069 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:10.752087 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:10.752115 (+ 28us) server_negotiation.cc:407] Connection header received
0926 16:41:10.752196 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:10.752202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:10.752472 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:10.752641 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:10.752993 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.753926 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.755548 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.756422 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.756555 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.756674 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:10.756769 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 16:41:10.756858 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1042,"thread_start_us":88,"threads_started":1}
I20260926 16:41:10.757743 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.751167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39107 (local address 127.0.0.1:60208)
0926 16:41:10.752902 (+ 1735us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:10.752918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:10.753001 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:10.753507 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:10.753510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:10.753529 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:10.753613 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.753619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.754933 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.754936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.756034 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:10.756043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.757188 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.757192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:10.757280 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.757314 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 16:41:10.757549 (+ 235us) client_negotiation.cc:240] Negotiation successful
0926 16:41:10.757618 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1664,"thread_start_us":95,"threads_started":1}
I20260926 16:41:10.757751 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.751557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.11.158.131:46281)
0926 16:41:10.752139 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:10.752174 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:10.752199 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:41:10.752276 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:10.752281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:10.752501 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:10.752659 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:10.753008 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.754212 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.755484 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.756056 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.756250 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.756420 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:10.757501 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0926 16:41:10.757614 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":142,"threads_started":1}
I20260926 16:41:10.758116 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:10.751955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.11.158.130:39107)
0926 16:41:10.752347 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:10.752358 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:10.753091 (+ 733us) server_negotiation.cc:407] Connection header received
0926 16:41:10.753171 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:10.753176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:10.753351 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:10.753492 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:10.753741 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.754801 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.756171 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:10.757070 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:10.757214 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:10.757329 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:10.757893 (+ 564us) server_negotiation.cc:299] Negotiation successful
0926 16:41:10.757988 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":171,"threads_started":1}
I20260926 16:41:10.760278 29054 tablet_service.cc:1467] Processing CreateTablet for tablet 8565a51ac6824fcfb8968849bd0aad2d (DEFAULT_TABLE table=test-table [id=7aed824a36f449e4821536b7ec8f31a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:10.760489 29188 tablet_service.cc:1467] Processing CreateTablet for tablet 8565a51ac6824fcfb8968849bd0aad2d (DEFAULT_TABLE table=test-table [id=7aed824a36f449e4821536b7ec8f31a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:10.760492 29322 tablet_service.cc:1467] Processing CreateTablet for tablet 8565a51ac6824fcfb8968849bd0aad2d (DEFAULT_TABLE table=test-table [id=7aed824a36f449e4821536b7ec8f31a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:10.761443 29054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8565a51ac6824fcfb8968849bd0aad2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:10.761471 29322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8565a51ac6824fcfb8968849bd0aad2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:10.761469 29188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8565a51ac6824fcfb8968849bd0aad2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:10.767894 29405 tablet_bootstrap.cc:492] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc: Bootstrap starting.
I20260926 16:41:10.769280 29407 tablet_bootstrap.cc:492] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Bootstrap starting.
I20260926 16:41:10.770255 29405 tablet_bootstrap.cc:654] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:10.771119 29407 tablet_bootstrap.cc:654] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:10.771468 29405 log.cc:824] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:10.771519 29406 tablet_bootstrap.cc:492] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: Bootstrap starting.
I20260926 16:41:10.773474 29406 tablet_bootstrap.cc:654] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:10.773640 29405 tablet_bootstrap.cc:492] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc: No bootstrap required, opened a new log
I20260926 16:41:10.773838 29405 ts_tablet_manager.cc:1397] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:41:10.774573 29406 log.cc:824] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:10.775990 29407 log.cc:824] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:10.776702 29406 tablet_bootstrap.cc:492] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: No bootstrap required, opened a new log
I20260926 16:41:10.776926 29406 ts_tablet_manager.cc:1397] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:41:10.777340 29405 raft_consensus.cc:348] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:10.777577 29405 raft_consensus.cc:374] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:10.777657 29405 raft_consensus.cc:729] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a06faa1eb934adda28e3ee327ddf3bc, State: Initialized, Role: FOLLOWER
I20260926 16:41:10.778061 29407 tablet_bootstrap.cc:492] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: No bootstrap required, opened a new log
I20260926 16:41:10.778149 29405 consensus_queue.cc:260] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:10.778245 29407 ts_tablet_manager.cc:1397] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260926 16:41:10.779193 29405 ts_tablet_manager.cc:1428] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:41:10.779562 29120 heartbeater.cc:503] Master 127.11.158.190:45771 was elected leader, sending a full tablet report...
I20260926 16:41:10.780809 29406 raft_consensus.cc:348] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:10.781071 29406 raft_consensus.cc:374] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:10.781172 29406 raft_consensus.cc:729] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345a7ad6d4514e209408c7817fb82652, State: Initialized, Role: FOLLOWER
I20260926 16:41:10.781673 29406 consensus_queue.cc:260] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:10.782069 29407 raft_consensus.cc:348] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:10.782356 29407 raft_consensus.cc:374] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:10.782366 29254 heartbeater.cc:503] Master 127.11.158.190:45771 was elected leader, sending a full tablet report...
I20260926 16:41:10.782464 29407 raft_consensus.cc:729] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb08123776de4522a0eaf1573747f698, State: Initialized, Role: FOLLOWER
I20260926 16:41:10.782687 29406 ts_tablet_manager.cc:1428] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:41:10.782943 29407 consensus_queue.cc:260] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:10.783643 29388 heartbeater.cc:503] Master 127.11.158.190:45771 was elected leader, sending a full tablet report...
I20260926 16:41:10.783936 29407 ts_tablet_manager.cc:1428] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260926 16:41:10.953405 29389 tablet.cc:2369] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:41:10.982097 29255 tablet.cc:2369] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:41:11.015534 29121 tablet.cc:2369] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:41:11.127970 29411 raft_consensus.cc:482] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:41:11.128178 29411 raft_consensus.cc:504] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:11.129364 29411 leader_election.cc:288] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb08123776de4522a0eaf1573747f698 (127.11.158.131:46281), 345a7ad6d4514e209408c7817fb82652 (127.11.158.130:39107)
I20260926 16:41:11.133733 29415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.129726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39107 (local address 127.11.158.129:36079)
0926 16:41:11.130139 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:11.130151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:11.130303 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:11.130596 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:11.130599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:11.130615 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:11.130755 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.130762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.131613 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.131617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.132546 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.132555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.133288 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.133292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.133390 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.133420 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 16:41:11.133498 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 16:41:11.133570 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":16,"thread_start_us":111,"threads_started":1}
I20260926 16:41:11.133774 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.129869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:36079 (local address 127.11.158.130:39107)
0926 16:41:11.130017 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:11.130025 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:11.130326 (+ 301us) server_negotiation.cc:407] Connection header received
0926 16:41:11.130411 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:11.130418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:11.130486 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:11.130582 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:11.130879 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.131496 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.132672 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.133168 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.133275 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.133328 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:11.133558 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 16:41:11.133635 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 16:41:11.134029 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.129839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:49707 (local address 127.11.158.131:46281)
0926 16:41:11.129978 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:11.129982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:11.130488 (+ 506us) server_negotiation.cc:407] Connection header received
0926 16:41:11.130624 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:11.130627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:11.130680 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:11.130760 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:11.131103 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.131625 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.132923 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.133392 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.133527 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.133572 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:11.133826 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 16:41:11.133926 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:41:11.134099 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.129726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:46281 (local address 127.11.158.129:49707)
0926 16:41:11.130375 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:11.130391 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:11.130500 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:11.130833 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:11.130838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:11.130861 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:11.130977 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.130984 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.131751 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.131754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.132793 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.132805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.133569 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.133573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.133681 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.133704 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:41:11.133791 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:41:11.133861 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":68,"thread_start_us":111,"threads_started":1}
I20260926 16:41:11.134809 29208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8565a51ac6824fcfb8968849bd0aad2d" candidate_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345a7ad6d4514e209408c7817fb82652" is_pre_election: true
I20260926 16:41:11.135154 29208 raft_consensus.cc:2454] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a06faa1eb934adda28e3ee327ddf3bc in term 0.
I20260926 16:41:11.135246 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8565a51ac6824fcfb8968849bd0aad2d" candidate_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb08123776de4522a0eaf1573747f698" is_pre_election: true
I20260926 16:41:11.135524 29342 raft_consensus.cc:2454] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a06faa1eb934adda28e3ee327ddf3bc in term 0.
I20260926 16:41:11.135664 29010 leader_election.cc:302] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345a7ad6d4514e209408c7817fb82652, 6a06faa1eb934adda28e3ee327ddf3bc; no voters:
I20260926 16:41:11.135973 29411 raft_consensus.cc:2781] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:41:11.136072 29411 raft_consensus.cc:482] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:41:11.136130 29411 raft_consensus.cc:3037] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:11.137219 29411 raft_consensus.cc:504] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:11.137575 29411 leader_election.cc:288] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [CANDIDATE]: Term 1 election: Requested vote from peers eb08123776de4522a0eaf1573747f698 (127.11.158.131:46281), 345a7ad6d4514e209408c7817fb82652 (127.11.158.130:39107)
I20260926 16:41:11.137969 29208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8565a51ac6824fcfb8968849bd0aad2d" candidate_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345a7ad6d4514e209408c7817fb82652"
I20260926 16:41:11.138073 29208 raft_consensus.cc:3037] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:11.139038 29208 raft_consensus.cc:2454] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a06faa1eb934adda28e3ee327ddf3bc in term 1.
I20260926 16:41:11.139293 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8565a51ac6824fcfb8968849bd0aad2d" candidate_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb08123776de4522a0eaf1573747f698"
I20260926 16:41:11.139338 29010 leader_election.cc:302] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345a7ad6d4514e209408c7817fb82652, 6a06faa1eb934adda28e3ee327ddf3bc; no voters:
I20260926 16:41:11.139422 29342 raft_consensus.cc:3037] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:11.139506 29411 raft_consensus.cc:2781] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:11.139736 29411 raft_consensus.cc:686] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [term 1 LEADER]: Becoming Leader. State: Replica: 6a06faa1eb934adda28e3ee327ddf3bc, State: Running, Role: LEADER
I20260926 16:41:11.140426 29411 consensus_queue.cc:237] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } }
I20260926 16:41:11.140728 29342 raft_consensus.cc:2454] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a06faa1eb934adda28e3ee327ddf3bc in term 1.
I20260926 16:41:11.143607 28929 catalog_manager.cc:5495] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a06faa1eb934adda28e3ee327ddf3bc (127.11.158.129). New cstate: current_term: 1 leader_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a06faa1eb934adda28e3ee327ddf3bc" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 46335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 } health_report { overall_health: UNKNOWN } } }
I20260926 16:41:11.208287 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.204710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.11.158.129:46335)
0926 16:41:11.204839 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:11.204843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:11.204859 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:41:11.204994 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:11.204997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:11.205055 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:11.205175 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:11.205438 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.205944 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.206820 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.207041 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.207154 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.207470 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:11.207886 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:11.208023 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:11.208083 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 16:41:11.208137 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:41:11.214365 29208 raft_consensus.cc:1261] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 1 FOLLOWER]: Refusing update from remote peer 6a06faa1eb934adda28e3ee327ddf3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:11.214365 29342 raft_consensus.cc:1261] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 1 FOLLOWER]: Refusing update from remote peer 6a06faa1eb934adda28e3ee327ddf3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:11.215065 29416 consensus_queue.cc:1035] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "345a7ad6d4514e209408c7817fb82652" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:11.215297 29411 consensus_queue.cc:1035] T 8565a51ac6824fcfb8968849bd0aad2d P 6a06faa1eb934adda28e3ee327ddf3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb08123776de4522a0eaf1573747f698" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:11.222559 29419 mvcc.cc:205] Tried to move back new op lower bound from 7333645808487141376 to 7333645808194117632. Current Snapshot: MvccSnapshot[applied={T|T < 7333645808487141376}]
I20260926 16:41:11.222838 29421 mvcc.cc:205] Tried to move back new op lower bound from 7333645808487141376 to 7333645808194117632. Current Snapshot: MvccSnapshot[applied={T|T < 7333645808487141376}]
I20260926 16:41:11.225872 29420 mvcc.cc:205] Tried to move back new op lower bound from 7333645808487141376 to 7333645808194117632. Current Snapshot: MvccSnapshot[applied={T|T < 7333645808487141376}]
I20260926 16:41:11.230962 28929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:11.231108 28929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:11.233624 28929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 810d1ee6fedb46bd9895a239752c1b1c: Sending DeleteTablet for 3 replicas of tablet 8565a51ac6824fcfb8968849bd0aad2d
I20260926 16:41:11.234393 29054 tablet_service.cc:1514] Processing DeleteTablet for tablet 8565a51ac6824fcfb8968849bd0aad2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:11 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:57554
I20260926 16:41:11.234704 29188 tablet_service.cc:1514] Processing DeleteTablet for tablet 8565a51ac6824fcfb8968849bd0aad2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:11 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:60208
I20260926 16:41:11.234905 29322 tablet_service.cc:1514] Processing DeleteTablet for tablet 8565a51ac6824fcfb8968849bd0aad2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:11 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:43856
I20260926 16:41:11.235224 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 28991
I20260926 16:41:11.236564 29430 tablet_replica.cc:330] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: stopping tablet replica
I20260926 16:41:11.236886 29430 raft_consensus.cc:2229] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:11.237191 29430 raft_consensus.cc:2258] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:11.238613 29430 ts_tablet_manager.cc:1905] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:11.240566 29429 tablet_replica.cc:330] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: stopping tablet replica
I20260926 16:41:11.240907 29429 raft_consensus.cc:2229] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:11.240957 29430 ts_tablet_manager.cc:1918] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:11.241078 29430 log.cc:1196] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/8565a51ac6824fcfb8968849bd0aad2d
I20260926 16:41:11.241225 29429 raft_consensus.cc:2258] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:11.241459 29430 ts_tablet_manager.cc:1939] T 8565a51ac6824fcfb8968849bd0aad2d P eb08123776de4522a0eaf1573747f698: Deleting consensus metadata
I20260926 16:41:11.242573 28915 catalog_manager.cc:4841] TS eb08123776de4522a0eaf1573747f698 (127.11.158.131:46281): tablet 8565a51ac6824fcfb8968849bd0aad2d (table test-table [id=7aed824a36f449e4821536b7ec8f31a8]) successfully deleted
I20260926 16:41:11.242774 29429 ts_tablet_manager.cc:1905] T 8565a51ac6824fcfb8968849bd0aad2d P 345a7ad6d4514e209408c7817fb82652: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:41:11.243049 28914 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:46335 (error 108)
I20260926 16:41:11.243299 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 29125
I20260926 16:41:11.244307 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.243948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:46335 (local address 127.0.0.1:57580)
0926 16:41:11.244128 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:11.244198 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 16:41:11.244483 28914 catalog_manager.cc:4584] TS 6a06faa1eb934adda28e3ee327ddf3bc (127.11.158.129:46335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8565a51ac6824fcfb8968849bd0aad2d: Network error: Client connection negotiation failed: client connection to 127.11.158.129:46335: connect: Connection refused (error 111)
I20260926 16:41:11.249540 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 29259
I20260926 16:41:11.251252 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.249711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:39107 (local address 127.0.0.1:60218)
0926 16:41:11.251071 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:11.251143 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.130:39107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1250}
I20260926 16:41:11.255081 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 28897
2026-09-26T16:41:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2658 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T16:41:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:11Z Disabled control of system clock
I20260926 16:41:11.286113 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:42575
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:42575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:41:11.399029 29435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:11.399325 29435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:11.399363 29435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:11.404461 29435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:11.404552 29435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:11.404569 29435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:11.404587 29435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:11.404603 29435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:11.409821 29435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:42575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:42575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:11.410897 29435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:11.412403 29435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:11.421218 29440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:11.421303 29443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:11.421365 29435 server_base.cc:1034] running on GCE node
W20260926 16:41:11.421207 29441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:11.422055 29435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:11.423199 29435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:11.424451 29435 hybrid_clock.cc:648] HybridClock initialized: now 1790440871424433 us; error 31 us; skew 500 ppm
I20260926 16:41:11.426311 29435 webserver.cc:460] Webserver started at http://127.11.158.190:43941/ using document root <none> and password file <none>
I20260926 16:41:11.426918 29435 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:11.427021 29435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:11.427264 29435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:11.429616 29435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "2f3a2f365252494b94683c55fa16e5de"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "ad8ff5a4a1d022116814a6f9c763c296"
server_key_iv: "88856681e2da76ed794263896ebbbd80"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.430248 29435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "2f3a2f365252494b94683c55fa16e5de"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "ad8ff5a4a1d022116814a6f9c763c296"
server_key_iv: "88856681e2da76ed794263896ebbbd80"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.434264 29435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:41:11.436802 29449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:11.438058 29435 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:11.438208 29435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "2f3a2f365252494b94683c55fa16e5de"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "ad8ff5a4a1d022116814a6f9c763c296"
server_key_iv: "88856681e2da76ed794263896ebbbd80"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.438336 29435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:11.451423 29435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:11.452188 29435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:11.452451 29435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:11.459040 29435 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:42575
I20260926 16:41:11.459045 29501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:42575 every 8 connection(s)
I20260926 16:41:11.459983 29435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:11.462283 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 29435
I20260926 16:41:11.462461 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:11.462760 11898 external_mini_cluster.cc:1428] Setting key 87a5df8e8bfa083b423e8cd3ed49e8bc
I20260926 16:41:11.463184 29502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:11.470067 29502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de: Bootstrap starting.
I20260926 16:41:11.472016 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.464112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.11.158.190:42575)
0926 16:41:11.464741 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:11.464757 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:11.464791 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:41:11.465430 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:11.465445 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:11.465815 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:11.466092 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:11.466558 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.467516 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.468760 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.469218 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.469334 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.470277 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:11.470282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:11.470294 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:11.470683 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:11.470740 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:11.470759 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:11.470842 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:11.471011 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 16:41:11.471302 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":177,"threads_started":1}
I20260926 16:41:11.472515 29502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:11.473309 29502 log.cc:824] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:11.475167 29502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de: No bootstrap required, opened a new log
I20260926 16:41:11.478533 29502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a2f365252494b94683c55fa16e5de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42575 } }
I20260926 16:41:11.478742 29502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:11.478776 29502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f3a2f365252494b94683c55fa16e5de, State: Initialized, Role: FOLLOWER
I20260926 16:41:11.479120 29502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a2f365252494b94683c55fa16e5de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42575 } }
I20260926 16:41:11.479223 29502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:11.479260 29502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:11.479326 29502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:11.480198 29502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a2f365252494b94683c55fa16e5de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42575 } }
I20260926 16:41:11.480598 29502 leader_election.cc:302] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f3a2f365252494b94683c55fa16e5de; no voters:
I20260926 16:41:11.480916 29502 leader_election.cc:288] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:11.481052 29507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:11.481287 29507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [term 1 LEADER]: Becoming Leader. State: Replica: 2f3a2f365252494b94683c55fa16e5de, State: Running, Role: LEADER
I20260926 16:41:11.481593 29507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a2f365252494b94683c55fa16e5de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42575 } }
I20260926 16:41:11.482098 29502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:11.483193 29509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f3a2f365252494b94683c55fa16e5de. Latest consensus state: current_term: 1 leader_uuid: "2f3a2f365252494b94683c55fa16e5de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a2f365252494b94683c55fa16e5de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42575 } } }
I20260926 16:41:11.483335 29509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:11.483649 29508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f3a2f365252494b94683c55fa16e5de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a2f365252494b94683c55fa16e5de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 42575 } } }
I20260926 16:41:11.483740 29508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:11.483839 29516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:11.486881 29516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:11.492211 29516 catalog_manager.cc:1347] Generated new cluster ID: dfb16ab9ff6b49438b43c21879626fae
I20260926 16:41:11.492280 29516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:11.509483 29516 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:11.510584 29516 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:11.521796 29516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de: Generated new TSK 0
I20260926 16:41:11.522672 29516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:11.533351 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42575
--builtin_ntp_servers=127.11.158.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:41:11.646222 29526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:11.646545 29526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:11.646626 29526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:11.650907 29526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:11.651005 29526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:11.651104 29526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:41:11.656030 29526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:11.657240 29526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:11.658634 29526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:11.668677 29526 server_base.cc:1034] running on GCE node
W20260926 16:41:11.668615 29532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:11.668601 29531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:11.668602 29534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:11.669332 29526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:11.670099 29526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:11.671273 29526 hybrid_clock.cc:648] HybridClock initialized: now 1790440871671237 us; error 56 us; skew 500 ppm
I20260926 16:41:11.673223 29526 webserver.cc:460] Webserver started at http://127.11.158.129:38783/ using document root <none> and password file <none>
I20260926 16:41:11.673830 29526 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:11.673893 29526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:11.674130 29526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:11.676177 29526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "b1892ebef6994408b7742a86338a0b0d"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "6a7a37f2f3961a180fd49c4d2e3fb05e"
server_key_iv: "ced40096ab2b4dd576b8b23135b7387e"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.676823 29526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "b1892ebef6994408b7742a86338a0b0d"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "6a7a37f2f3961a180fd49c4d2e3fb05e"
server_key_iv: "ced40096ab2b4dd576b8b23135b7387e"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.680495 29526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:11.683053 29540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:11.684471 29526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:11.684590 29526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "b1892ebef6994408b7742a86338a0b0d"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "6a7a37f2f3961a180fd49c4d2e3fb05e"
server_key_iv: "ced40096ab2b4dd576b8b23135b7387e"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.684712 29526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:11.696979 29526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:11.697710 29526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:11.697928 29526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:11.698567 29526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:11.699553 29526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:11.699601 29526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:11.699642 29526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:11.699698 29526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:11.708858 29526 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:34629
I20260926 16:41:11.708885 29653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:34629 every 8 connection(s)
I20260926 16:41:11.709810 29526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:41:11.719936 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.711774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:34985 (local address 127.11.158.190:42575)
0926 16:41:11.711946 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:11.711949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:11.713042 (+ 1093us) server_negotiation.cc:407] Connection header received
0926 16:41:11.714467 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:11.714470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:11.714545 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:11.714661 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:11.715583 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.716596 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.717909 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.718156 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.718276 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.719361 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:11.719365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:11.719368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:11.719382 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:11.719407 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:11.719420 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:11.719521 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:11.719688 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 16:41:11.719781 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 16:41:11.720135 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 29526
I20260926 16:41:11.720265 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:41:11.720606 11898 external_mini_cluster.cc:1428] Setting key 40501dd8d9bc303225feb66704159a74
I20260926 16:41:11.721015 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.712167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:42575 (local address 127.11.158.129:34985)
0926 16:41:11.712872 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:11.712914 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:11.714240 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:11.714842 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:11.714849 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:11.714935 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:11.715427 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.715438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.716795 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.716801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.717767 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.717776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.718322 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.718326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.718503 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.719092 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:11.719093 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:11.719134 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:11.719199 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:11.719524 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:11.719536 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:11.719544 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 16:41:11.719688 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 16:41:11.719970 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":157,"threads_started":1}
I20260926 16:41:11.722800 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42575
--builtin_ntp_servers=127.11.158.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:41:11.723402 29654 heartbeater.cc:347] Connected to a master server at 127.11.158.190:42575
I20260926 16:41:11.723657 29654 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:11.724323 29654 heartbeater.cc:511] Master 127.11.158.190:42575 requested a full tablet report, sending...
I20260926 16:41:11.726018 29465 ts_manager.cc:194] Registered new tserver with Master: b1892ebef6994408b7742a86338a0b0d (127.11.158.129:34629)
I20260926 16:41:11.727716 29465 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.129:34985
W20260926 16:41:11.841105 29657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:11.841394 29657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:11.841434 29657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:11.845825 29657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:11.845907 29657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:11.845968 29657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:41:11.851326 29657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:11.852582 29657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:11.854039 29657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:11.864871 29665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:11.864898 29663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:11.864984 29657 server_base.cc:1034] running on GCE node
W20260926 16:41:11.865245 29662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:11.865688 29657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:11.866361 29657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:11.867528 29657 hybrid_clock.cc:648] HybridClock initialized: now 1790440871867489 us; error 51 us; skew 500 ppm
I20260926 16:41:11.869508 29657 webserver.cc:460] Webserver started at http://127.11.158.130:38895/ using document root <none> and password file <none>
I20260926 16:41:11.870332 29657 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:11.870433 29657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:11.870675 29657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:11.873683 29657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "2d73c63edb3c44448553119b570c3315"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "134fb95ada1cf1675607ff89dbc510a6"
server_key_iv: "1c081ef5a9220340084ab6e2f8053fad"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.874293 29657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "2d73c63edb3c44448553119b570c3315"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "134fb95ada1cf1675607ff89dbc510a6"
server_key_iv: "1c081ef5a9220340084ab6e2f8053fad"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.878410 29657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 16:41:11.881106 29671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:11.882534 29657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:41:11.882653 29657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "2d73c63edb3c44448553119b570c3315"
format_stamp: "Formatted at 2026-09-26 16:41:11 on dist-test-slave-p39x"
server_key: "134fb95ada1cf1675607ff89dbc510a6"
server_key_iv: "1c081ef5a9220340084ab6e2f8053fad"
server_key_version: "encryptionkey@0"
I20260926 16:41:11.882823 29657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:11.897223 29657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:11.897957 29657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:11.898128 29657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:11.898828 29657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:11.899997 29657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:11.900053 29657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:11.900095 29657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:11.900111 29657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:11.910030 29657 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:38981
I20260926 16:41:11.910048 29784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:38981 every 8 connection(s)
I20260926 16:41:11.911089 29657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:41:11.920796 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.913048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:50157 (local address 127.11.158.190:42575)
0926 16:41:11.913201 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:11.913206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:11.914367 (+ 1161us) server_negotiation.cc:407] Connection header received
0926 16:41:11.915754 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:11.915757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:11.915829 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:11.915913 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:11.917113 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.917639 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.918828 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.919066 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.919160 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.920298 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:11.920301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:11.920303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:11.920318 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:11.920364 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:11.920374 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:11.920436 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:11.920579 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 16:41:11.920650 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:41:11.921319 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 29657
I20260926 16:41:11.921433 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:41:11.921725 11898 external_mini_cluster.cc:1428] Setting key 39659370f036db4d7c2dd5a3f1ef3a8c
I20260926 16:41:11.922027 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:11.913551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:42575 (local address 127.11.158.130:50157)
0926 16:41:11.914214 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:11.914256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:11.915528 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:11.916128 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:11.916135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:11.916225 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:11.916932 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.916963 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.917773 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.917776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.918688 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:11.918697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:11.919218 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:11.919221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:11.919365 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:11.920027 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:11.920029 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:11.920071 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:11.920139 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:11.920447 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:11.920454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:11.920459 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 16:41:11.920560 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 16:41:11.920818 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":118,"threads_started":1}
I20260926 16:41:11.923292 29785 heartbeater.cc:347] Connected to a master server at 127.11.158.190:42575
I20260926 16:41:11.923586 29785 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:11.923975 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42575
--builtin_ntp_servers=127.11.158.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:41:11.924235 29785 heartbeater.cc:511] Master 127.11.158.190:42575 requested a full tablet report, sending...
I20260926 16:41:11.925482 29465 ts_manager.cc:194] Registered new tserver with Master: 2d73c63edb3c44448553119b570c3315 (127.11.158.130:38981)
I20260926 16:41:11.926131 29465 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.130:50157
W20260926 16:41:12.037439 29788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:12.037710 29788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:12.037751 29788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:12.042132 29788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:12.042198 29788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:12.042258 29788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:41:12.047318 29788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:12.048527 29788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:12.049959 29788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:12.060758 29793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:12.061012 29796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:12.061041 29788 server_base.cc:1034] running on GCE node
W20260926 16:41:12.060773 29794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:12.061551 29788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:12.062312 29788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:12.063527 29788 hybrid_clock.cc:648] HybridClock initialized: now 1790440872063487 us; error 58 us; skew 500 ppm
I20260926 16:41:12.065727 29788 webserver.cc:460] Webserver started at http://127.11.158.131:43707/ using document root <none> and password file <none>
I20260926 16:41:12.066395 29788 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:12.066493 29788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:12.066736 29788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:12.069083 29788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "d8dcf71b98f94a1f87e66287c96f92a8"
format_stamp: "Formatted at 2026-09-26 16:41:12 on dist-test-slave-p39x"
server_key: "840eae5db6a4f50ab766d5fda046348a"
server_key_iv: "1a683da842e7f2572c6ddf18718e8ee7"
server_key_version: "encryptionkey@0"
I20260926 16:41:12.069691 29788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "d8dcf71b98f94a1f87e66287c96f92a8"
format_stamp: "Formatted at 2026-09-26 16:41:12 on dist-test-slave-p39x"
server_key: "840eae5db6a4f50ab766d5fda046348a"
server_key_iv: "1a683da842e7f2572c6ddf18718e8ee7"
server_key_version: "encryptionkey@0"
I20260926 16:41:12.074162 29788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:41:12.076921 29802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:12.078373 29788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:12.078544 29788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "d8dcf71b98f94a1f87e66287c96f92a8"
format_stamp: "Formatted at 2026-09-26 16:41:12 on dist-test-slave-p39x"
server_key: "840eae5db6a4f50ab766d5fda046348a"
server_key_iv: "1a683da842e7f2572c6ddf18718e8ee7"
server_key_version: "encryptionkey@0"
I20260926 16:41:12.078675 29788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:12.104238 29788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:12.105051 29788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:12.105269 29788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:12.105907 29788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:12.106868 29788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:12.106949 29788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:12.107040 29788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:12.107089 29788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:12.116255 29788 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:34313
I20260926 16:41:12.116278 29915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:34313 every 8 connection(s)
I20260926 16:41:12.117201 29788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:41:12.122331 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 29788
I20260926 16:41:12.122417 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:41:12.122802 11898 external_mini_cluster.cc:1428] Setting key ae2484779c8edf209d4cffd78a6c1ea0
I20260926 16:41:12.126682 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.118980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:58801 (local address 127.11.158.190:42575)
0926 16:41:12.119138 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.119142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.120274 (+ 1132us) server_negotiation.cc:407] Connection header received
0926 16:41:12.121671 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.121674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.121725 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.121807 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:12.122799 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.123514 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.124778 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.125011 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.125122 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.126162 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:12.126165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:12.126169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:12.126181 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:12.126209 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:12.126217 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:12.126291 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.126478 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.126541 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:41:12.127552 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.119459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:42575 (local address 127.11.158.131:58801)
0926 16:41:12.120125 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.120168 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:12.121433 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:12.121976 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:12.121983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:12.122078 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:12.122650 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.122666 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.123647 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.123651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.124654 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.124664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.125162 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.125166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.125335 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.125900 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:12.125901 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:12.125945 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:12.126012 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:12.126306 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:12.126317 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:12.126327 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 16:41:12.126480 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 16:41:12.126774 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":107,"threads_started":1}
I20260926 16:41:12.128711 29916 heartbeater.cc:347] Connected to a master server at 127.11.158.190:42575
I20260926 16:41:12.128968 29916 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:12.129504 29916 heartbeater.cc:511] Master 127.11.158.190:42575 requested a full tablet report, sending...
I20260926 16:41:12.130532 29465 ts_manager.cc:194] Registered new tserver with Master: d8dcf71b98f94a1f87e66287c96f92a8 (127.11.158.131:34313)
I20260926 16:41:12.131105 29465 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.131:58801
I20260926 16:41:12.137007 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:41:12.143123 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.139079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.11.158.190:42575)
0926 16:41:12.139204 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.139208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.139363 (+ 155us) server_negotiation.cc:407] Connection header received
0926 16:41:12.139487 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.139489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.139531 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.139612 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:12.139911 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.140537 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.141464 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.141740 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.141823 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.142735 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:12.142741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:12.142744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:12.142759 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:12.142787 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:12.142796 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:12.142854 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.142956 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.143007 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:41:12.146340 29465 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:41:12.149828 29465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:41:12.164490 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.158532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.11.158.129:34629)
0926 16:41:12.159620 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.159632 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.159660 (+ 28us) server_negotiation.cc:407] Connection header received
0926 16:41:12.159740 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.159745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.159969 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.160500 (+ 531us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:12.160636 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.161872 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.163232 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.163774 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.163918 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.164112 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.164195 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.164291 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":975,"thread_start_us":95,"threads_started":1}
I20260926 16:41:12.164489 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.158415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34629 (local address 127.0.0.1:41790)
0926 16:41:12.158838 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.158894 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:12.159021 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:12.160143 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:12.160149 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:12.160171 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:12.160322 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.160360 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.162007 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.162011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.163110 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.163119 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.163908 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.163912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.164026 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.164062 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 16:41:12.164170 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 16:41:12.164292 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":4,"thread_start_us":91,"threads_started":1}
I20260926 16:41:12.164759 29931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.159247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.11.158.130:38981)
0926 16:41:12.159761 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.159772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.159791 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:41:12.160105 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.160110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.160281 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.160499 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:12.160728 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.161863 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.163312 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.164073 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.164259 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.164441 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.164488 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.164572 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":114,"threads_started":1}
I20260926 16:41:12.165055 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.158466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38981 (local address 127.0.0.1:58678)
0926 16:41:12.159245 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.159258 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:12.159346 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:12.160501 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:12.160504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:12.160518 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:12.160616 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.160622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.162007 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.162011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.163187 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.163195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.164197 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.164200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.164305 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.164326 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:41:12.164435 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 16:41:12.164510 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":72,"thread_start_us":118,"threads_started":1}
I20260926 16:41:12.165596 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.158954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:34313 (local address 127.0.0.1:51884)
0926 16:41:12.159272 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.159284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:12.159375 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:12.160311 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:12.160314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:12.160329 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:12.160501 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.160507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.161554 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.161557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.162923 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.162933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.164196 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.164200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.165287 (+ 1087us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.165311 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:41:12.165400 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:41:12.165472 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":69,"thread_start_us":93,"threads_started":1}
I20260926 16:41:12.166095 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.159254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.11.158.131:34313)
0926 16:41:12.159698 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.159712 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.159740 (+ 28us) server_negotiation.cc:407] Connection header received
0926 16:41:12.159813 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.159818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.159983 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.160154 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:12.160634 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.161429 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.163054 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.163841 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.163967 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.164135 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.165451 (+ 1316us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.165939 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":191,"threads_started":1}
I20260926 16:41:12.167764 29588 tablet_service.cc:1467] Processing CreateTablet for tablet f40e723be3ae4261b3b54eeebaf90da1 (DEFAULT_TABLE table=test-table [id=7c2b49e81e304a48b3a836dfbacb1156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:12.167912 29719 tablet_service.cc:1467] Processing CreateTablet for tablet f40e723be3ae4261b3b54eeebaf90da1 (DEFAULT_TABLE table=test-table [id=7c2b49e81e304a48b3a836dfbacb1156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:12.168830 29588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f40e723be3ae4261b3b54eeebaf90da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:12.168938 29719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f40e723be3ae4261b3b54eeebaf90da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:12.168687 29850 tablet_service.cc:1467] Processing CreateTablet for tablet f40e723be3ae4261b3b54eeebaf90da1 (DEFAULT_TABLE table=test-table [id=7c2b49e81e304a48b3a836dfbacb1156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:12.169606 29850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f40e723be3ae4261b3b54eeebaf90da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:12.175453 29933 tablet_bootstrap.cc:492] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: Bootstrap starting.
I20260926 16:41:12.176515 29934 tablet_bootstrap.cc:492] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: Bootstrap starting.
I20260926 16:41:12.176888 29935 tablet_bootstrap.cc:492] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Bootstrap starting.
I20260926 16:41:12.177984 29933 tablet_bootstrap.cc:654] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:12.178573 29935 tablet_bootstrap.cc:654] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:12.178606 29934 tablet_bootstrap.cc:654] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:12.178869 29933 log.cc:824] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:12.179265 29935 log.cc:824] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:12.179415 29934 log.cc:824] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:12.181118 29935 tablet_bootstrap.cc:492] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: No bootstrap required, opened a new log
I20260926 16:41:12.181118 29933 tablet_bootstrap.cc:492] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: No bootstrap required, opened a new log
I20260926 16:41:12.181322 29933 ts_tablet_manager.cc:1397] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:41:12.181322 29935 ts_tablet_manager.cc:1397] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:41:12.185822 29934 tablet_bootstrap.cc:492] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: No bootstrap required, opened a new log
I20260926 16:41:12.186069 29934 ts_tablet_manager.cc:1397] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260926 16:41:12.185886 29933 raft_consensus.cc:348] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.185883 29935 raft_consensus.cc:348] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.186167 29935 raft_consensus.cc:374] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:12.186167 29933 raft_consensus.cc:374] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:12.186226 29935 raft_consensus.cc:729] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8dcf71b98f94a1f87e66287c96f92a8, State: Initialized, Role: FOLLOWER
I20260926 16:41:12.186235 29933 raft_consensus.cc:729] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1892ebef6994408b7742a86338a0b0d, State: Initialized, Role: FOLLOWER
I20260926 16:41:12.186784 29935 consensus_queue.cc:260] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.186784 29933 consensus_queue.cc:260] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.187909 29933 ts_tablet_manager.cc:1428] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 16:41:12.187909 29935 ts_tablet_manager.cc:1428] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 16:41:12.188274 29916 heartbeater.cc:503] Master 127.11.158.190:42575 was elected leader, sending a full tablet report...
I20260926 16:41:12.188274 29654 heartbeater.cc:503] Master 127.11.158.190:42575 was elected leader, sending a full tablet report...
I20260926 16:41:12.190666 29934 raft_consensus.cc:348] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.190992 29934 raft_consensus.cc:374] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:12.191061 29934 raft_consensus.cc:729] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d73c63edb3c44448553119b570c3315, State: Initialized, Role: FOLLOWER
I20260926 16:41:12.191617 29934 consensus_queue.cc:260] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.192714 29934 ts_tablet_manager.cc:1428] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260926 16:41:12.193143 29785 heartbeater.cc:503] Master 127.11.158.190:42575 was elected leader, sending a full tablet report...
W20260926 16:41:12.211156 29655 tablet.cc:2369] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:41:12.211169 29940 raft_consensus.cc:482] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:41:12.211418 29940 raft_consensus.cc:504] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.212656 29940 leader_election.cc:288] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1892ebef6994408b7742a86338a0b0d (127.11.158.129:34629), 2d73c63edb3c44448553119b570c3315 (127.11.158.130:38981)
I20260926 16:41:12.216358 29931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.213101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:34239 (local address 127.11.158.130:38981)
0926 16:41:12.213221 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.213225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.213240 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:41:12.213348 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.213351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.213403 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.213494 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:12.213777 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.214284 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.215355 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.215767 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.215906 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.215957 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.216134 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.216204 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:41:12.216328 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.212966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38981 (local address 127.11.158.131:34239)
0926 16:41:12.213100 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.213118 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:12.213232 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:12.213508 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:12.213511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:12.213529 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:12.213656 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.213663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.214420 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.214427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.215217 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.215225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.215903 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.215907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.216006 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.216027 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:41:12.216119 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:41:12.216191 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260926 16:41:12.216856 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.212974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34629 (local address 127.11.158.131:34727)
0926 16:41:12.213480 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.213493 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:12.213624 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:12.213899 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:12.213901 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:12.213917 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:12.214042 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.214048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.214777 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.214782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.215568 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:12.215576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.216230 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.216233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:12.216375 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.216399 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:41:12.216634 (+ 235us) client_negotiation.cc:240] Negotiation successful
0926 16:41:12.216702 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":157,"thread_start_us":169,"threads_started":1}
I20260926 16:41:12.216884 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.213077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:34727 (local address 127.11.158.129:34629)
0926 16:41:12.213205 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.213212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.213587 (+ 375us) server_negotiation.cc:407] Connection header received
0926 16:41:12.213745 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.213749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.213800 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.213888 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:12.214158 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.214660 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.215699 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.216099 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.216230 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.216276 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.216656 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.216737 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:41:12.217101 29739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40e723be3ae4261b3b54eeebaf90da1" candidate_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d73c63edb3c44448553119b570c3315" is_pre_election: true
I20260926 16:41:12.217438 29739 raft_consensus.cc:2454] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8dcf71b98f94a1f87e66287c96f92a8 in term 0.
I20260926 16:41:12.217458 29608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40e723be3ae4261b3b54eeebaf90da1" candidate_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1892ebef6994408b7742a86338a0b0d" is_pre_election: true
I20260926 16:41:12.217796 29608 raft_consensus.cc:2454] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8dcf71b98f94a1f87e66287c96f92a8 in term 0.
I20260926 16:41:12.217983 29804 leader_election.cc:302] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d73c63edb3c44448553119b570c3315, d8dcf71b98f94a1f87e66287c96f92a8; no voters:
I20260926 16:41:12.218338 29940 raft_consensus.cc:2781] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:41:12.218438 29940 raft_consensus.cc:482] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:41:12.218495 29940 raft_consensus.cc:3037] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:12.219655 29940 raft_consensus.cc:504] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.220060 29940 leader_election.cc:288] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [CANDIDATE]: Term 1 election: Requested vote from peers b1892ebef6994408b7742a86338a0b0d (127.11.158.129:34629), 2d73c63edb3c44448553119b570c3315 (127.11.158.130:38981)
I20260926 16:41:12.220407 29608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40e723be3ae4261b3b54eeebaf90da1" candidate_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1892ebef6994408b7742a86338a0b0d"
I20260926 16:41:12.220422 29739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40e723be3ae4261b3b54eeebaf90da1" candidate_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d73c63edb3c44448553119b570c3315"
I20260926 16:41:12.220531 29608 raft_consensus.cc:3037] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:12.220574 29739 raft_consensus.cc:3037] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:12.221642 29608 raft_consensus.cc:2454] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8dcf71b98f94a1f87e66287c96f92a8 in term 1.
I20260926 16:41:12.221647 29739 raft_consensus.cc:2454] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8dcf71b98f94a1f87e66287c96f92a8 in term 1.
I20260926 16:41:12.221985 29804 leader_election.cc:302] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d73c63edb3c44448553119b570c3315, d8dcf71b98f94a1f87e66287c96f92a8; no voters:
I20260926 16:41:12.222188 29940 raft_consensus.cc:2781] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:12.222386 29940 raft_consensus.cc:686] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 1 LEADER]: Becoming Leader. State: Replica: d8dcf71b98f94a1f87e66287c96f92a8, State: Running, Role: LEADER
I20260926 16:41:12.222771 29940 consensus_queue.cc:237] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } }
I20260926 16:41:12.226004 29465 catalog_manager.cc:5495] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8dcf71b98f94a1f87e66287c96f92a8 (127.11.158.131). New cstate: current_term: 1 leader_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8dcf71b98f94a1f87e66287c96f92a8" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34313 } health_report { overall_health: HEALTHY } } }
I20260926 16:41:12.244047 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.240571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.11.158.131:34313)
0926 16:41:12.240702 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:12.240706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:12.240719 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:41:12.240864 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:12.240867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:12.240919 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:12.241015 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:12.241213 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.241791 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.242611 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:12.242871 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:12.242953 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:12.243235 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:12.243672 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:12.243765 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:12.243854 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 16:41:12.243905 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:41:12.249820 29608 raft_consensus.cc:1261] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 1 FOLLOWER]: Refusing update from remote peer d8dcf71b98f94a1f87e66287c96f92a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:12.249814 29739 raft_consensus.cc:1261] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 1 FOLLOWER]: Refusing update from remote peer d8dcf71b98f94a1f87e66287c96f92a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:12.250546 29940 consensus_queue.cc:1035] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1892ebef6994408b7742a86338a0b0d" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:12.250746 29943 consensus_queue.cc:1035] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d73c63edb3c44448553119b570c3315" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:12.258302 29946 mvcc.cc:205] Tried to move back new op lower bound from 7333645812728156160 to 7333645812627582976. Current Snapshot: MvccSnapshot[applied={T|T < 7333645812728156160}]
I20260926 16:41:12.260027 29947 mvcc.cc:205] Tried to move back new op lower bound from 7333645812728156160 to 7333645812627582976. Current Snapshot: MvccSnapshot[applied={T|T < 7333645812728156160}]
I20260926 16:41:12.269594 29465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:12.269804 29465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:12.272614 29465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f3a2f365252494b94683c55fa16e5de: Sending DeleteTablet for 3 replicas of tablet f40e723be3ae4261b3b54eeebaf90da1
I20260926 16:41:12.273483 29588 tablet_service.cc:1514] Processing DeleteTablet for tablet f40e723be3ae4261b3b54eeebaf90da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:12 UTC) from {username='slave'} at 127.0.0.1:41790
I20260926 16:41:12.273589 29719 tablet_service.cc:1514] Processing DeleteTablet for tablet f40e723be3ae4261b3b54eeebaf90da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:12 UTC) from {username='slave'} at 127.0.0.1:58678
I20260926 16:41:12.273744 29850 tablet_service.cc:1514] Processing DeleteTablet for tablet f40e723be3ae4261b3b54eeebaf90da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:12 UTC) from {username='slave'} at 127.0.0.1:51884
I20260926 16:41:12.274214 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 29526
I20260926 16:41:12.274221 29956 tablet_replica.cc:330] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d: stopping tablet replica
I20260926 16:41:12.274560 29956 raft_consensus.cc:2229] T f40e723be3ae4261b3b54eeebaf90da1 P b1892ebef6994408b7742a86338a0b0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:12.274824 29957 tablet_replica.cc:330] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: stopping tablet replica
I20260926 16:41:12.275117 29957 raft_consensus.cc:2229] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:12.275475 29957 raft_consensus.cc:2258] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:12.281810 29958 tablet_replica.cc:330] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: stopping tablet replica
I20260926 16:41:12.282189 29958 raft_consensus.cc:2229] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 1 LEADER]: Raft consensus shutting down.
W20260926 16:41:12.282660 29453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:34629 (error 108)
I20260926 16:41:12.282773 29958 raft_consensus.cc:2258] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:12.283412 29957 ts_tablet_manager.cc:1905] T f40e723be3ae4261b3b54eeebaf90da1 P 2d73c63edb3c44448553119b570c3315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:12.283746 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 29657
I20260926 16:41:12.285085 29958 ts_tablet_manager.cc:1905] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:12.286558 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.286166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34629 (local address 127.0.0.1:41802)
0926 16:41:12.286368 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.286445 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:34629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260926 16:41:12.286755 29453 catalog_manager.cc:4584] TS b1892ebef6994408b7742a86338a0b0d (127.11.158.129:34629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f40e723be3ae4261b3b54eeebaf90da1: Network error: Client connection negotiation failed: client connection to 127.11.158.129:34629: connect: Connection refused (error 111)
I20260926 16:41:12.288044 29958 ts_tablet_manager.cc:1918] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:12.288164 29958 log.cc:1196] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/f40e723be3ae4261b3b54eeebaf90da1
I20260926 16:41:12.288569 29958 ts_tablet_manager.cc:1939] T f40e723be3ae4261b3b54eeebaf90da1 P d8dcf71b98f94a1f87e66287c96f92a8: Deleting consensus metadata
I20260926 16:41:12.289686 29450 catalog_manager.cc:4841] TS d8dcf71b98f94a1f87e66287c96f92a8 (127.11.158.131:34313): tablet f40e723be3ae4261b3b54eeebaf90da1 (table test-table [id=7c2b49e81e304a48b3a836dfbacb1156]) successfully deleted
I20260926 16:41:12.291733 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 29788
I20260926 16:41:12.293655 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:12.292026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:38981 (local address 127.0.0.1:58686)
0926 16:41:12.293493 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:12.293558 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.130:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1354}
I20260926 16:41:12.296972 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 29435
2026-09-26T16:41:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1040 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:12 dist-test-slave-p39x krb5kdc[29962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:12 dist-test-slave-p39x krb5kdc[29962](info): set up 2 sockets
Sep 26 16:41:12 dist-test-slave-p39x krb5kdc[29962](info): commencing operation
krb5kdc: starting...
W20260926 16:41:13.344828 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:13 dist-test-slave-p39x krb5kdc[29962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:13.467666 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:46671
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:46671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:13.578343 29975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:13.578624 29975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:13.578665 29975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:13.582578 29975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:13.582631 29975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:13.582650 29975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:13.582667 29975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:13.582682 29975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:13.587878 29975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:46671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:46671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:13.588940 29975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:13.589927 29975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:13.598490 29980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:13.598507 29981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:13.598466 29975 server_base.cc:1034] running on GCE node
W20260926 16:41:13.598531 29983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:13.599332 29975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:13.600515 29975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:13.601722 29975 hybrid_clock.cc:648] HybridClock initialized: now 1790440873601703 us; error 36 us; skew 500 ppm
Sep 26 16:41:13 dist-test-slave-p39x krb5kdc[29962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440873, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:13.604657 29975 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:13.605512 29975 webserver.cc:460] Webserver started at http://127.11.158.190:33277/ using document root <none> and password file <none>
I20260926 16:41:13.606033 29975 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:13.606087 29975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:13.606246 29975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:13.608234 29975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "5c984511971c4a8db521ce75134f6cec"
format_stamp: "Formatted at 2026-09-26 16:41:13 on dist-test-slave-p39x"
server_key: "388f86e9d4a378621552bb3a8c4a43ef"
server_key_iv: "dce67161cbd0eebb1a0041479fd4b031"
server_key_version: "encryptionkey@0"
I20260926 16:41:13.608830 29975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "5c984511971c4a8db521ce75134f6cec"
format_stamp: "Formatted at 2026-09-26 16:41:13 on dist-test-slave-p39x"
server_key: "388f86e9d4a378621552bb3a8c4a43ef"
server_key_iv: "dce67161cbd0eebb1a0041479fd4b031"
server_key_version: "encryptionkey@0"
I20260926 16:41:13.612574 29975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:41:13.614991 29990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:13.616416 29975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:13.616532 29975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "5c984511971c4a8db521ce75134f6cec"
format_stamp: "Formatted at 2026-09-26 16:41:13 on dist-test-slave-p39x"
server_key: "388f86e9d4a378621552bb3a8c4a43ef"
server_key_iv: "dce67161cbd0eebb1a0041479fd4b031"
server_key_version: "encryptionkey@0"
I20260926 16:41:13.616624 29975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:13.628456 29975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:13.631764 29975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:13.631920 29975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:13.639161 29975 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:46671
I20260926 16:41:13.639243 30042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:46671 every 8 connection(s)
I20260926 16:41:13.640175 29975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:13.643198 30043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:13.644264 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 29975
I20260926 16:41:13.644443 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:13.644786 11898 external_mini_cluster.cc:1428] Setting key 12a5acc3fe8952483f789110a66069c5
I20260926 16:41:13.649811 30043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec: Bootstrap starting.
I20260926 16:41:13.652565 30043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:13.653504 30043 log.cc:824] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:41:13 dist-test-slave-p39x krb5kdc[29962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:13.655481 30043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec: No bootstrap required, opened a new log
I20260926 16:41:13.659739 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:13.646184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.11.158.190:46671)
0926 16:41:13.646585 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:13.646601 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:13.646636 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:41:13.647267 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:13.647283 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:13.647617 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:13.647893 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:13.648840 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:13.649817 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:13.650967 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:13.651413 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:13.651543 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:13.654574 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:13.654579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:13.654593 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:13.654618 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:13.656636 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:13.657067 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:13.657073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:13.657079 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:13.657214 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:13.657479 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:13.657482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:13.657484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:13.658048 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:13.658257 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:13.658483 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 16:41:13.658782 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":124,"threads_started":1}
I20260926 16:41:13.660159 30043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c984511971c4a8db521ce75134f6cec" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46671 } }
I20260926 16:41:13.660422 30043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:13.660508 30043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c984511971c4a8db521ce75134f6cec, State: Initialized, Role: FOLLOWER
I20260926 16:41:13.660925 30043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c984511971c4a8db521ce75134f6cec" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46671 } }
I20260926 16:41:13.661083 30043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:13.661171 30043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:13.661268 30043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:13.662159 30043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c984511971c4a8db521ce75134f6cec" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46671 } }
I20260926 16:41:13.662495 30043 leader_election.cc:302] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c984511971c4a8db521ce75134f6cec; no voters:
I20260926 16:41:13.662794 30043 leader_election.cc:288] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:13.662935 30048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:13.663165 30048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [term 1 LEADER]: Becoming Leader. State: Replica: 5c984511971c4a8db521ce75134f6cec, State: Running, Role: LEADER
I20260926 16:41:13.663452 30048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c984511971c4a8db521ce75134f6cec" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46671 } }
I20260926 16:41:13.664023 30043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:13.665241 30049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c984511971c4a8db521ce75134f6cec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c984511971c4a8db521ce75134f6cec" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46671 } } }
I20260926 16:41:13.665383 30049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:13.665666 30050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c984511971c4a8db521ce75134f6cec. Latest consensus state: current_term: 1 leader_uuid: "5c984511971c4a8db521ce75134f6cec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c984511971c4a8db521ce75134f6cec" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 46671 } } }
I20260926 16:41:13.665756 30050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:13.665803 30057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:13.668746 30057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:13.674396 30057 catalog_manager.cc:1347] Generated new cluster ID: 18d996bdb6c9461ab572c7c71ad7e46d
I20260926 16:41:13.674472 30057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:13.682075 30057 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:13.683060 30057 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:13.687649 30057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec: Generated new TSK 0
I20260926 16:41:13.688391 30057 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:41:13.720669 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46671
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:13.833005 30069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:13.833292 30069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:13.833343 30069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:13.837580 30069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:13.837647 30069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:13.837735 30069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:41:13.844615 30069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:13.846226 30069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:13.847617 30069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:13.858121 30075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:13.858143 30074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:13.858302 30069 server_base.cc:1034] running on GCE node
W20260926 16:41:13.858148 30077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:13.858762 30069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:13.859453 30069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:13.860700 30069 hybrid_clock.cc:648] HybridClock initialized: now 1790440873860680 us; error 33 us; skew 500 ppm
Sep 26 16:41:13 dist-test-slave-p39x krb5kdc[29962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440873, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:13.863620 30069 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:41:13.864728 30069 webserver.cc:460] Webserver started at http://127.11.158.129:41379/ using document root <none> and password file <none>
I20260926 16:41:13.865274 30069 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:13.865360 30069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:13.865625 30069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:13.867797 30069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "f50579e81f96461ebcb184b69c87b146"
format_stamp: "Formatted at 2026-09-26 16:41:13 on dist-test-slave-p39x"
server_key: "49732991a05e004e09b1776e00eb852d"
server_key_iv: "b3169af5b39e7ff4eadd3abbdd7a8a73"
server_key_version: "encryptionkey@0"
I20260926 16:41:13.868460 30069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "f50579e81f96461ebcb184b69c87b146"
format_stamp: "Formatted at 2026-09-26 16:41:13 on dist-test-slave-p39x"
server_key: "49732991a05e004e09b1776e00eb852d"
server_key_iv: "b3169af5b39e7ff4eadd3abbdd7a8a73"
server_key_version: "encryptionkey@0"
I20260926 16:41:13.872496 30069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:41:13.875164 30084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:13.876703 30069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:13.876849 30069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "f50579e81f96461ebcb184b69c87b146"
format_stamp: "Formatted at 2026-09-26 16:41:13 on dist-test-slave-p39x"
server_key: "49732991a05e004e09b1776e00eb852d"
server_key_iv: "b3169af5b39e7ff4eadd3abbdd7a8a73"
server_key_version: "encryptionkey@0"
I20260926 16:41:13.876986 30069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:13.902700 30069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:13.907505 30069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:13.907680 30069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:13.908299 30069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:13.909291 30069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:13.909358 30069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:13.909422 30069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:13.909449 30069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:13.921478 30069 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:34887
I20260926 16:41:13.921538 30197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:34887 every 8 connection(s)
I20260926 16:41:13.922613 30069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:41:13.927855 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 30069
I20260926 16:41:13.928025 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:41:13.928395 11898 external_mini_cluster.cc:1428] Setting key 635903bb8a742a64239b5d442ac1af07
Sep 26 16:41:13 dist-test-slave-p39x krb5kdc[29962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440873, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:41:13.943038 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:13.924660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:50585 (local address 127.11.158.190:46671)
0926 16:41:13.926217 (+ 1557us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:13.926221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:13.926304 (+ 83us) server_negotiation.cc:407] Connection header received
0926 16:41:13.928954 (+ 2650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:13.928958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:13.929032 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:13.929123 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:13.930905 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:13.931720 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:13.933996 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:13.934472 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:13.934570 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:13.938436 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:13.938444 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:13.938448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:13.938490 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:13.940309 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:13.940994 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:13.940998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:13.941000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:13.941086 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:13.941471 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:13.941474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:13.941476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:13.941680 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:13.941792 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:13.942626 (+ 834us) server_negotiation.cc:299] Negotiation successful
0926 16:41:13.942800 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1465}
I20260926 16:41:13.944192 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:13.925249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:46671 (local address 127.11.158.129:50585)
0926 16:41:13.926098 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:13.926185 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:13.928628 (+ 2443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:13.929373 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:13.929387 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:13.929959 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:13.930709 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:13.930730 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:13.931900 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:13.931905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:13.933817 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:13.933835 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:13.934832 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:13.934839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:13.935104 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:13.935938 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:13.935967 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:13.938172 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:13.940487 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:13.940494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:13.940513 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:13.940875 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:13.941190 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:13.941193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:13.941196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:13.941360 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:13.941789 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:13.941836 (+ 47us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:13.942243 (+ 407us) client_negotiation.cc:769] Sending connection context
0926 16:41:13.942524 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 16:41:13.942943 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":138,"threads_started":1}
I20260926 16:41:13.945592 30198 heartbeater.cc:347] Connected to a master server at 127.11.158.190:46671
I20260926 16:41:13.945885 30198 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:13.946575 30198 heartbeater.cc:511] Master 127.11.158.190:46671 requested a full tablet report, sending...
I20260926 16:41:13.948515 30007 ts_manager.cc:194] Registered new tserver with Master: f50579e81f96461ebcb184b69c87b146 (127.11.158.129:34887)
I20260926 16:41:13.950124 30007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:50585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:41:13.963579 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46671
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:14.077730 30203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:14.078027 30203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:14.078071 30203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:14.082177 30203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:14.082242 30203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:14.082300 30203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:41:14.088001 30203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:14.089493 30203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:14.090560 30203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:14.100275 30209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:14.100291 30211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:14.100418 30203 server_base.cc:1034] running on GCE node
W20260926 16:41:14.100630 30208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:14.101105 30203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:14.101809 30203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:14.102975 30203 hybrid_clock.cc:648] HybridClock initialized: now 1790440874102947 us; error 40 us; skew 500 ppm
Sep 26 16:41:14 dist-test-slave-p39x krb5kdc[29962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:14.106729 30203 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:41:14.108834 30203 webserver.cc:460] Webserver started at http://127.11.158.130:36495/ using document root <none> and password file <none>
I20260926 16:41:14.109427 30203 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:14.109490 30203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:14.109663 30203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:14.111901 30203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "ba25d6e8d11d5019b2a34cf9e05abeff"
server_key_iv: "13f85c3ab24897a8a835bf89cd332234"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.112543 30203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "ba25d6e8d11d5019b2a34cf9e05abeff"
server_key_iv: "13f85c3ab24897a8a835bf89cd332234"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.116463 30203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:41:14.118911 30218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.120266 30203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:41:14.120450 30203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "ba25d6e8d11d5019b2a34cf9e05abeff"
server_key_iv: "13f85c3ab24897a8a835bf89cd332234"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.120548 30203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:14.133523 30203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:14.137041 30203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:14.137212 30203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:14.137774 30203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:14.138579 30203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:14.138625 30203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.138664 30203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:14.138679 30203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.147747 30203 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:37445
I20260926 16:41:14.147771 30331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:37445 every 8 connection(s)
I20260926 16:41:14.148706 30203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:41:14.150704 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 30203
I20260926 16:41:14.150862 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:41:14.151206 11898 external_mini_cluster.cc:1428] Setting key 900ffcc2fb377a33988966d3ca7094d5
Sep 26 16:41:14 dist-test-slave-p39x krb5kdc[29962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
I20260926 16:41:14.166455 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.150649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:37823 (local address 127.11.158.190:46671)
0926 16:41:14.150837 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.150843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.151790 (+ 947us) server_negotiation.cc:407] Connection header received
0926 16:41:14.153390 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.153392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.153436 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.153503 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:14.155060 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.155954 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.157914 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.158137 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.158230 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.161348 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:14.161356 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:14.161362 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:14.161410 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:14.163018 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:14.163552 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:14.163555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:14.163556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:14.163635 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:14.164010 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:14.164013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:14.164015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:14.164212 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:14.164376 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.164975 (+ 599us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.165128 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:41:14.166435 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.151051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:46671 (local address 127.11.158.130:37823)
0926 16:41:14.151640 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.151684 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.153152 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.153697 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.153708 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.154230 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:14.154905 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.154920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.156099 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.156106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.157771 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.157785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.158312 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.158318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.158582 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.159341 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:14.159369 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:14.161157 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:14.163142 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:14.163147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:14.163161 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:14.163447 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:14.163722 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:14.163725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:14.163726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:14.163874 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:14.164370 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:14.164380 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:14.164693 (+ 313us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.164913 (+ 220us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.165261 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":128,"threads_started":1}
I20260926 16:41:14.167929 30332 heartbeater.cc:347] Connected to a master server at 127.11.158.190:46671
I20260926 16:41:14.168195 30332 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:14.168772 30332 heartbeater.cc:511] Master 127.11.158.190:46671 requested a full tablet report, sending...
I20260926 16:41:14.169960 30007 ts_manager.cc:194] Registered new tserver with Master: a9ccd14c7bea4b3ea09d9d15ca763f45 (127.11.158.130:37445)
I20260926 16:41:14.170571 30007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:37823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:41:14.183615 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46671
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:14.295778 30337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:14.296065 30337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:14.296126 30337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:14.300307 30337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:14.300423 30337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:14.300510 30337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:41:14.305847 30337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:46671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:14.306893 30337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:14.307914 30337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:14.317332 30342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:14.317368 30343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:14.317504 30337 server_base.cc:1034] running on GCE node
W20260926 16:41:14.317339 30345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:14.318136 30337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:14.318852 30337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:14.320055 30337 hybrid_clock.cc:648] HybridClock initialized: now 1790440874320026 us; error 50 us; skew 500 ppm
Sep 26 16:41:14 dist-test-slave-p39x krb5kdc[29962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:14.322942 30337 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:41:14.323928 30337 webserver.cc:460] Webserver started at http://127.11.158.131:45069/ using document root <none> and password file <none>
I20260926 16:41:14.324617 30337 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:14.324712 30337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:14.324941 30337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:14.327031 30337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "55f0c628684e4d3299d1742be0a4e9e4"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "4ded16ade29e06f2c185e5ae7429ce0b"
server_key_iv: "c1dc8d0d0013705fa17200eda4a0992f"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.327627 30337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "55f0c628684e4d3299d1742be0a4e9e4"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "4ded16ade29e06f2c185e5ae7429ce0b"
server_key_iv: "c1dc8d0d0013705fa17200eda4a0992f"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.331496 30337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:41:14.334069 30352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.335253 30337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:41:14.335389 30337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "55f0c628684e4d3299d1742be0a4e9e4"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "4ded16ade29e06f2c185e5ae7429ce0b"
server_key_iv: "c1dc8d0d0013705fa17200eda4a0992f"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.335505 30337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:14.349002 30337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:14.351873 30337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:14.352089 30337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:14.352691 30337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:14.353489 30337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:14.353559 30337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.353631 30337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:14.353672 30337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.361919 30337 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:39579
I20260926 16:41:14.361953 30465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:39579 every 8 connection(s)
I20260926 16:41:14.362824 30337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:41:14.370013 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 30337
I20260926 16:41:14.370183 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:41:14.370489 11898 external_mini_cluster.cc:1428] Setting key 67c73c87c8b42cd8ebafcf845e03e421
Sep 26 16:41:14 dist-test-slave-p39x krb5kdc[29962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:14.377081 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.364716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:38941 (local address 127.11.158.190:46671)
0926 16:41:14.364867 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.364872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.366286 (+ 1414us) server_negotiation.cc:407] Connection header received
0926 16:41:14.367140 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.367142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.367190 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.367263 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:14.368448 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.368960 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.370098 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.370437 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.370553 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.373616 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:14.373620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:14.373623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:14.373650 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:14.374903 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:14.375421 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:14.375426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:14.375428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:14.375515 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:14.375883 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:14.375887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:14.375888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:14.376090 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:14.376200 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.376784 (+ 584us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.376915 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 16:41:14.377905 30468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.365168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:46671 (local address 127.11.158.131:38941)
0926 16:41:14.366130 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.366173 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.366953 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.367430 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.367437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.367776 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:14.368264 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.368277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.369072 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.369074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.369956 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.369966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.370593 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.370597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.370811 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.371499 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:14.371524 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:14.373446 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:14.375028 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:14.375034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:14.375050 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:14.375313 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:14.375610 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:14.375615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:14.375617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:14.375762 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:14.376208 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:14.376217 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:14.376534 (+ 317us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.376728 (+ 194us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.377067 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":591,"threads_started":1}
I20260926 16:41:14.379001 30466 heartbeater.cc:347] Connected to a master server at 127.11.158.190:46671
I20260926 16:41:14.379253 30466 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:14.379757 30466 heartbeater.cc:511] Master 127.11.158.190:46671 requested a full tablet report, sending...
I20260926 16:41:14.380856 30007 ts_manager.cc:194] Registered new tserver with Master: 55f0c628684e4d3299d1742be0a4e9e4 (127.11.158.131:39579)
I20260926 16:41:14.381449 30007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:38941
I20260926 16:41:14.384719 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:41:14.394217 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.386659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.11.158.190:46671)
0926 16:41:14.386815 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.386819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.387028 (+ 209us) server_negotiation.cc:407] Connection header received
0926 16:41:14.387090 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.387092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.387131 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.387199 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:14.387833 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.388330 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.389177 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.389385 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.389460 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.391016 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:14.391018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:14.391021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:14.391044 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:14.392428 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:14.392910 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:14.392913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:14.392914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:14.392988 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:14.393341 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:14.393344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:14.393346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:14.393582 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:14.393708 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.393929 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.394043 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:41:14.397266 30007 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:41:14.400238 30007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:41:14.414681 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.409158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.11.158.130:37445)
0926 16:41:14.409534 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.409552 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.409578 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:41:14.410002 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.410009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.410208 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.410714 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:14.410875 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.411965 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.413422 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.413995 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.414137 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.414275 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.414424 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.414532 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":89,"threads_started":1}
I20260926 16:41:14.414678 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.408791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:37445 (local address 127.0.0.1:52774)
0926 16:41:14.409357 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.409413 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.409534 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.410380 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.410386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.410421 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:14.410570 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.410578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.412136 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.412140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.413275 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.413285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.414122 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.414126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.414234 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.414271 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.414389 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.414527 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":43,"spinlock_wait_cycles":3456,"thread_start_us":85,"threads_started":1}
I20260926 16:41:14.414934 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.408777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.11.158.131:39579)
0926 16:41:14.409821 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.409832 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.409855 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:41:14.409991 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.409996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.410206 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.410723 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:14.410845 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.411547 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.413242 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.414119 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.414544 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.414660 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.414709 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.414795 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":122,"threads_started":1}
I20260926 16:41:14.414916 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.409336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34887 (local address 127.0.0.1:53242)
0926 16:41:14.409744 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.409757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.409838 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.410554 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.410557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.410570 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:14.410651 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.410661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.412092 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.412095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.413307 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.413313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.414544 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.414547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.414624 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.414644 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.414715 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.414778 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":25,"thread_start_us":73,"threads_started":1}
I20260926 16:41:14.415087 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.408630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:39579 (local address 127.0.0.1:37846)
0926 16:41:14.409758 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.409774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.409860 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.410386 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.410388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.410423 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:14.410570 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.410577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.411679 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.411684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.413113 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.413124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.414269 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.414275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.414365 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.414386 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.414436 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.414879 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":428,"spinlock_wait_cycles":9216,"thread_start_us":90,"threads_started":1}
I20260926 16:41:14.415421 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.409509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.11.158.129:34887)
0926 16:41:14.410115 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.410131 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.410151 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:41:14.410233 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.410238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.410413 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.410561 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:14.410743 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.411930 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.413451 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.414243 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.414360 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.414487 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.415094 (+ 607us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.415225 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":84,"threads_started":1}
I20260926 16:41:14.418110 30266 tablet_service.cc:1467] Processing CreateTablet for tablet b8706cd039814bf6a813650a1c7313a9 (DEFAULT_TABLE table=test-table [id=8209e3cbc8954bf09c330b17a5c548c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:14.418140 30132 tablet_service.cc:1467] Processing CreateTablet for tablet b8706cd039814bf6a813650a1c7313a9 (DEFAULT_TABLE table=test-table [id=8209e3cbc8954bf09c330b17a5c548c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:14.418140 30400 tablet_service.cc:1467] Processing CreateTablet for tablet b8706cd039814bf6a813650a1c7313a9 (DEFAULT_TABLE table=test-table [id=8209e3cbc8954bf09c330b17a5c548c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:14.419121 30266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8706cd039814bf6a813650a1c7313a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:14.419126 30132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8706cd039814bf6a813650a1c7313a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:14.419126 30400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8706cd039814bf6a813650a1c7313a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:14.425716 30483 tablet_bootstrap.cc:492] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: Bootstrap starting.
I20260926 16:41:14.426433 30485 tablet_bootstrap.cc:492] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Bootstrap starting.
I20260926 16:41:14.428606 30485 tablet_bootstrap.cc:654] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:14.428606 30483 tablet_bootstrap.cc:654] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:14.428695 30484 tablet_bootstrap.cc:492] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146: Bootstrap starting.
I20260926 16:41:14.430081 30483 log.cc:824] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:14.430074 30485 log.cc:824] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:14.430593 30484 tablet_bootstrap.cc:654] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:14.431896 30484 log.cc:824] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:14.432564 30483 tablet_bootstrap.cc:492] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: No bootstrap required, opened a new log
I20260926 16:41:14.432600 30485 tablet_bootstrap.cc:492] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: No bootstrap required, opened a new log
I20260926 16:41:14.432796 30485 ts_tablet_manager.cc:1397] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 16:41:14.432811 30483 ts_tablet_manager.cc:1397] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:41:14.434010 30484 tablet_bootstrap.cc:492] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146: No bootstrap required, opened a new log
I20260926 16:41:14.434199 30484 ts_tablet_manager.cc:1397] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:41:14.435935 30485 raft_consensus.cc:348] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.436293 30485 raft_consensus.cc:374] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:14.436410 30485 raft_consensus.cc:729] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f0c628684e4d3299d1742be0a4e9e4, State: Initialized, Role: FOLLOWER
I20260926 16:41:14.436673 30483 raft_consensus.cc:348] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.436934 30483 raft_consensus.cc:374] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:14.437000 30483 raft_consensus.cc:729] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ccd14c7bea4b3ea09d9d15ca763f45, State: Initialized, Role: FOLLOWER
I20260926 16:41:14.436951 30485 consensus_queue.cc:260] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.437429 30483 consensus_queue.cc:260] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.438052 30485 ts_tablet_manager.cc:1428] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:41:14.438066 30484 raft_consensus.cc:348] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.438300 30484 raft_consensus.cc:374] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:14.438401 30484 raft_consensus.cc:729] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50579e81f96461ebcb184b69c87b146, State: Initialized, Role: FOLLOWER
I20260926 16:41:14.438422 30466 heartbeater.cc:503] Master 127.11.158.190:46671 was elected leader, sending a full tablet report...
I20260926 16:41:14.438853 30484 consensus_queue.cc:260] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.439201 30332 heartbeater.cc:503] Master 127.11.158.190:46671 was elected leader, sending a full tablet report...
I20260926 16:41:14.439291 30483 ts_tablet_manager.cc:1428] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:41:14.439610 30490 raft_consensus.cc:482] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:41:14.439731 30490 raft_consensus.cc:504] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.439783 30484 ts_tablet_manager.cc:1428] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:41:14.440085 30198 heartbeater.cc:503] Master 127.11.158.190:46671 was elected leader, sending a full tablet report...
I20260926 16:41:14.441015 30490 leader_election.cc:288] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55f0c628684e4d3299d1742be0a4e9e4 (127.11.158.131:39579), f50579e81f96461ebcb184b69c87b146 (127.11.158.129:34887)
I20260926 16:41:14.446202 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.441249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34887 (local address 127.11.158.130:53017)
0926 16:41:14.441440 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.441461 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.441562 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.442557 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.442561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.442581 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:14.442703 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.442710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.443845 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.443848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.445131 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.445140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.445774 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.445778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.445881 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.445903 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.445998 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.446073 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260926 16:41:14.446202 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.441327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:53017 (local address 127.11.158.129:34887)
0926 16:41:14.441434 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.441438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.441545 (+ 107us) server_negotiation.cc:407] Connection header received
0926 16:41:14.441725 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.441728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.441775 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.441842 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:14.442805 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.443519 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.445257 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.445648 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.445786 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.445843 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.446012 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.446081 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:41:14.446977 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.442469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:38747 (local address 127.11.158.131:39579)
0926 16:41:14.442581 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.442587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.442834 (+ 247us) server_negotiation.cc:407] Connection header received
0926 16:41:14.443014 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.443017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.443062 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.443109 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:14.443642 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.444305 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.445860 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.446288 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.446423 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.446471 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.446764 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.446861 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 16:41:14.447321 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.441735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:39579 (local address 127.11.158.130:38747)
0926 16:41:14.442753 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.442768 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.442895 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.443270 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.443273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.443290 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:14.443518 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.443525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.444675 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.444678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.445728 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.445738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.446534 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.446537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.446607 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.446624 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.447109 (+ 485us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.447185 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"thread_start_us":111,"threads_started":1}
I20260926 16:41:14.447265 30139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8706cd039814bf6a813650a1c7313a9" candidate_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50579e81f96461ebcb184b69c87b146" is_pre_election: true
I20260926 16:41:14.447578 30139 raft_consensus.cc:2454] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ccd14c7bea4b3ea09d9d15ca763f45 in term 0.
I20260926 16:41:14.448015 30219 leader_election.cc:302] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ccd14c7bea4b3ea09d9d15ca763f45, f50579e81f96461ebcb184b69c87b146; no voters:
I20260926 16:41:14.448016 30420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8706cd039814bf6a813650a1c7313a9" candidate_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f0c628684e4d3299d1742be0a4e9e4" is_pre_election: true
I20260926 16:41:14.448294 30420 raft_consensus.cc:2454] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ccd14c7bea4b3ea09d9d15ca763f45 in term 0.
I20260926 16:41:14.448307 30490 raft_consensus.cc:2781] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:41:14.448443 30490 raft_consensus.cc:482] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:41:14.448518 30490 raft_consensus.cc:3037] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:14.449568 30490 raft_consensus.cc:504] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.449942 30490 leader_election.cc:288] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [CANDIDATE]: Term 1 election: Requested vote from peers 55f0c628684e4d3299d1742be0a4e9e4 (127.11.158.131:39579), f50579e81f96461ebcb184b69c87b146 (127.11.158.129:34887)
I20260926 16:41:14.450230 30420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8706cd039814bf6a813650a1c7313a9" candidate_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f0c628684e4d3299d1742be0a4e9e4"
I20260926 16:41:14.450286 30139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8706cd039814bf6a813650a1c7313a9" candidate_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50579e81f96461ebcb184b69c87b146"
I20260926 16:41:14.450354 30420 raft_consensus.cc:3037] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:14.450397 30139 raft_consensus.cc:3037] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:14.451350 30139 raft_consensus.cc:2454] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ccd14c7bea4b3ea09d9d15ca763f45 in term 1.
I20260926 16:41:14.451432 30420 raft_consensus.cc:2454] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ccd14c7bea4b3ea09d9d15ca763f45 in term 1.
I20260926 16:41:14.451635 30219 leader_election.cc:302] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ccd14c7bea4b3ea09d9d15ca763f45, f50579e81f96461ebcb184b69c87b146; no voters:
I20260926 16:41:14.451792 30490 raft_consensus.cc:2781] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:14.451982 30490 raft_consensus.cc:686] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 1 LEADER]: Becoming Leader. State: Replica: a9ccd14c7bea4b3ea09d9d15ca763f45, State: Running, Role: LEADER
I20260926 16:41:14.452328 30490 consensus_queue.cc:237] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } }
I20260926 16:41:14.455567 30001 catalog_manager.cc:5495] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ccd14c7bea4b3ea09d9d15ca763f45 (127.11.158.130). New cstate: current_term: 1 leader_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ccd14c7bea4b3ea09d9d15ca763f45" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 } health_report { overall_health: UNKNOWN } } }
I20260926 16:41:14.466962 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.463140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.11.158.130:37445)
0926 16:41:14.463256 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.463259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.463271 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:41:14.463384 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.463386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.463436 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.463540 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:14.463800 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.464278 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.465346 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.465602 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.465683 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.466164 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:14.466577 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:14.466688 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.466800 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.466835 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:41:14.473076 30420 raft_consensus.cc:1261] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 1 FOLLOWER]: Refusing update from remote peer a9ccd14c7bea4b3ea09d9d15ca763f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:14.473076 30139 raft_consensus.cc:1261] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 1 FOLLOWER]: Refusing update from remote peer a9ccd14c7bea4b3ea09d9d15ca763f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:14.473853 30493 consensus_queue.cc:1035] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f50579e81f96461ebcb184b69c87b146" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 34887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:14.474043 30490 consensus_queue.cc:1035] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55f0c628684e4d3299d1742be0a4e9e4" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 39579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:14.489952 30005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:14.490099 30005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:14.492473 30005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c984511971c4a8db521ce75134f6cec: Sending DeleteTablet for 3 replicas of tablet b8706cd039814bf6a813650a1c7313a9
I20260926 16:41:14.493211 30400 tablet_service.cc:1514] Processing DeleteTablet for tablet b8706cd039814bf6a813650a1c7313a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:14 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:37846
I20260926 16:41:14.493347 30132 tablet_service.cc:1514] Processing DeleteTablet for tablet b8706cd039814bf6a813650a1c7313a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:14 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:53242
I20260926 16:41:14.493692 30505 tablet_replica.cc:330] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: stopping tablet replica
I20260926 16:41:14.493809 30266 tablet_service.cc:1514] Processing DeleteTablet for tablet b8706cd039814bf6a813650a1c7313a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:14 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:52774
I20260926 16:41:14.494119 30505 raft_consensus.cc:2229] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:14.494261 30506 tablet_replica.cc:330] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146: stopping tablet replica
I20260926 16:41:14.494468 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 30069
I20260926 16:41:14.494544 30506 raft_consensus.cc:2229] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:14.494823 30506 raft_consensus.cc:2258] T b8706cd039814bf6a813650a1c7313a9 P f50579e81f96461ebcb184b69c87b146 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:14.495635 30507 tablet_replica.cc:330] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: stopping tablet replica
I20260926 16:41:14.495904 30507 raft_consensus.cc:2229] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:41:14.494575 30505 raft_consensus.cc:2258] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:14.496263 30507 raft_consensus.cc:2258] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:14.497797 30505 ts_tablet_manager.cc:1905] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:14.500015 30505 ts_tablet_manager.cc:1918] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:14.500128 30505 log.cc:1196] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/b8706cd039814bf6a813650a1c7313a9
I20260926 16:41:14.500553 30505 ts_tablet_manager.cc:1939] T b8706cd039814bf6a813650a1c7313a9 P 55f0c628684e4d3299d1742be0a4e9e4: Deleting consensus metadata
I20260926 16:41:14.501425 30507 ts_tablet_manager.cc:1905] T b8706cd039814bf6a813650a1c7313a9 P a9ccd14c7bea4b3ea09d9d15ca763f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:14.501812 29992 catalog_manager.cc:4841] TS 55f0c628684e4d3299d1742be0a4e9e4 (127.11.158.131:39579): tablet b8706cd039814bf6a813650a1c7313a9 (table test-table [id=8209e3cbc8954bf09c330b17a5c548c5]) successfully deleted
W20260926 16:41:14.502151 29991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:34887 (error 108)
I20260926 16:41:14.502988 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 30203
I20260926 16:41:14.503753 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.502896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:34887 (local address 127.0.0.1:53256)
0926 16:41:14.503600 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.503669 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:34887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607}
W20260926 16:41:14.508519 29991 catalog_manager.cc:4584] TS f50579e81f96461ebcb184b69c87b146 (127.11.158.129:34887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8706cd039814bf6a813650a1c7313a9: Network error: Client connection negotiation failed: client connection to 127.11.158.129:34887: connect: Connection refused (error 111)
I20260926 16:41:14.508708 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 30337
I20260926 16:41:14.510094 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.509791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:37445 (local address 127.0.0.1:52782)
0926 16:41:14.509925 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.509987 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.130:37445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 16:41:14.514851 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 29975
2026-09-26T16:41:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2218 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T16:41:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:14Z Disabled control of system clock
I20260926 16:41:14.545085 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38615
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:41:14.655058 30512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:14.655331 30512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:14.655367 30512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:14.659385 30512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:14.659458 30512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:14.659474 30512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:14.659492 30512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:14.659507 30512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:14.664485 30512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:14.665663 30512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:14.667097 30512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:14.675899 30520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:14.675863 30518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:14.675963 30512 server_base.cc:1034] running on GCE node
W20260926 16:41:14.675863 30517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:14.676931 30512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:14.678180 30512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:14.679389 30512 hybrid_clock.cc:648] HybridClock initialized: now 1790440874679372 us; error 33 us; skew 500 ppm
I20260926 16:41:14.681351 30512 webserver.cc:460] Webserver started at http://127.11.158.190:45215/ using document root <none> and password file <none>
I20260926 16:41:14.681946 30512 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:14.682040 30512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:14.682274 30512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:14.684388 30512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "2797a742daf34e59bc2ff6366ca717de"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "2f4b2f3f22a4dfc2059479f9c66c73fa"
server_key_iv: "b5c52bbb0d8302574c30bda1b546817b"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.684999 30512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "2797a742daf34e59bc2ff6366ca717de"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "2f4b2f3f22a4dfc2059479f9c66c73fa"
server_key_iv: "b5c52bbb0d8302574c30bda1b546817b"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.688894 30512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:41:14.691356 30526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.692879 30512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:14.693034 30512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "2797a742daf34e59bc2ff6366ca717de"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "2f4b2f3f22a4dfc2059479f9c66c73fa"
server_key_iv: "b5c52bbb0d8302574c30bda1b546817b"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.693156 30512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:14.714218 30512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:14.715031 30512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:14.715260 30512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:14.722252 30512 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:38615
I20260926 16:41:14.722265 30578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:38615 every 8 connection(s)
I20260926 16:41:14.723238 30512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:14.726720 30579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:14.730980 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 30512
I20260926 16:41:14.731139 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:14.731467 11898 external_mini_cluster.cc:1428] Setting key 05610515088ef5e82fbe53d3ec4659d0
I20260926 16:41:14.733011 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de: Bootstrap starting.
I20260926 16:41:14.735848 30579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:14.736786 30579 log.cc:824] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:14.738874 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de: No bootstrap required, opened a new log
I20260926 16:41:14.742549 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.733038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.11.158.190:38615)
0926 16:41:14.733758 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.733779 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.733831 (+ 52us) server_negotiation.cc:407] Connection header received
0926 16:41:14.734719 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.734743 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.735270 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.735635 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:14.736147 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.737564 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.738499 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.739185 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.739342 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.740275 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:14.740281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:14.740300 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:14.740846 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:14.740923 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:14.740949 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:14.741049 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.741211 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.741489 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":113,"threads_started":1}
I20260926 16:41:14.743392 30579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2797a742daf34e59bc2ff6366ca717de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38615 } }
I20260926 16:41:14.743680 30579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:14.743738 30579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2797a742daf34e59bc2ff6366ca717de, State: Initialized, Role: FOLLOWER
I20260926 16:41:14.744239 30579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2797a742daf34e59bc2ff6366ca717de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38615 } }
I20260926 16:41:14.744443 30579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:14.744534 30579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:14.744632 30579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:14.745566 30579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2797a742daf34e59bc2ff6366ca717de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38615 } }
I20260926 16:41:14.745915 30579 leader_election.cc:302] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2797a742daf34e59bc2ff6366ca717de; no voters:
I20260926 16:41:14.746225 30579 leader_election.cc:288] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:14.746364 30584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:14.746599 30584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [term 1 LEADER]: Becoming Leader. State: Replica: 2797a742daf34e59bc2ff6366ca717de, State: Running, Role: LEADER
I20260926 16:41:14.746932 30584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2797a742daf34e59bc2ff6366ca717de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38615 } }
I20260926 16:41:14.747606 30579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:14.748795 30586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2797a742daf34e59bc2ff6366ca717de. Latest consensus state: current_term: 1 leader_uuid: "2797a742daf34e59bc2ff6366ca717de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2797a742daf34e59bc2ff6366ca717de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38615 } } }
I20260926 16:41:14.748842 30585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2797a742daf34e59bc2ff6366ca717de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2797a742daf34e59bc2ff6366ca717de" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38615 } } }
I20260926 16:41:14.748929 30586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:14.748929 30585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:14.749316 30593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:14.752414 30593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:14.757519 30593 catalog_manager.cc:1347] Generated new cluster ID: fdd95a9f40084450adcbde34eaf415b0
I20260926 16:41:14.757591 30593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:14.764863 30593 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:14.765933 30593 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:14.775380 30593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de: Generated new TSK 0
I20260926 16:41:14.776106 30593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:14.782727 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38615
--builtin_ntp_servers=127.11.158.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:41:14.898885 30603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:14.899135 30603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:14.899174 30603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:14.903184 30603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:14.903249 30603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:14.903306 30603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:41:14.908160 30603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:14.909323 30603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:14.910748 30603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:14.920725 30611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:14.920764 30609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:14.920713 30608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:14.920931 30603 server_base.cc:1034] running on GCE node
I20260926 16:41:14.921511 30603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:14.922175 30603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:14.923367 30603 hybrid_clock.cc:648] HybridClock initialized: now 1790440874923323 us; error 65 us; skew 500 ppm
I20260926 16:41:14.925381 30603 webserver.cc:460] Webserver started at http://127.11.158.129:42273/ using document root <none> and password file <none>
I20260926 16:41:14.926010 30603 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:14.926110 30603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:14.926374 30603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:14.928650 30603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "0cefe510553c4e0bbdfce46b182e3e22"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "720a57807593e7fe445dbdea375c39f7"
server_key_iv: "b66e7a461ae50f9f5ee31c731c3fff81"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.929301 30603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "0cefe510553c4e0bbdfce46b182e3e22"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "720a57807593e7fe445dbdea375c39f7"
server_key_iv: "b66e7a461ae50f9f5ee31c731c3fff81"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.933298 30603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:41:14.935753 30617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.937034 30603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:14.937167 30603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "0cefe510553c4e0bbdfce46b182e3e22"
format_stamp: "Formatted at 2026-09-26 16:41:14 on dist-test-slave-p39x"
server_key: "720a57807593e7fe445dbdea375c39f7"
server_key_iv: "b66e7a461ae50f9f5ee31c731c3fff81"
server_key_version: "encryptionkey@0"
I20260926 16:41:14.937279 30603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:14.963621 30603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:14.964421 30603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:14.964624 30603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:14.965219 30603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:14.966159 30603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:14.966230 30603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.966302 30603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:14.966346 30603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:14.975029 30603 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:42513
I20260926 16:41:14.975083 30730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:42513 every 8 connection(s)
I20260926 16:41:14.975924 30603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:41:14.979341 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 30603
I20260926 16:41:14.979478 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:41:14.979765 11898 external_mini_cluster.cc:1428] Setting key 58207daa5fb9cdd46e7797c01d7613dd
I20260926 16:41:14.981775 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38615
--builtin_ntp_servers=127.11.158.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:41:14.986423 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.977745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:53781 (local address 127.11.158.190:38615)
0926 16:41:14.977900 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:14.977905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:14.978908 (+ 1003us) server_negotiation.cc:407] Connection header received
0926 16:41:14.980471 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:14.980474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:14.980550 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:14.980634 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:14.981638 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.982478 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.984183 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.984500 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.984593 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.985886 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:14.985890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:14.985894 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:14.985915 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:14.985935 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:14.985944 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:14.986022 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:14.986190 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 16:41:14.986266 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 16:41:14.987301 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:14.978122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38615 (local address 127.11.158.129:53781)
0926 16:41:14.978762 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:14.978802 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:14.980176 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:14.980795 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:14.980802 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:14.980887 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:14.981492 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.981507 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.982626 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.982629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.984031 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:14.984045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:14.984646 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:14.984649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:14.984944 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:14.985600 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:14.985601 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:14.985644 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:14.985712 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:14.986024 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:14.986036 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:14.986043 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:41:14.986179 (+ 136us) client_negotiation.cc:240] Negotiation successful
0926 16:41:14.986476 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":112,"threads_started":1}
I20260926 16:41:14.988616 30731 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38615
I20260926 16:41:14.988946 30731 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:14.989501 30731 heartbeater.cc:511] Master 127.11.158.190:38615 requested a full tablet report, sending...
I20260926 16:41:14.991238 30543 ts_manager.cc:194] Registered new tserver with Master: 0cefe510553c4e0bbdfce46b182e3e22 (127.11.158.129:42513)
I20260926 16:41:14.993068 30543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.129:53781
W20260926 16:41:15.094125 30734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:15.094363 30734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:15.094408 30734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:15.098304 30734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:15.098347 30734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:15.098403 30734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:41:15.102834 30734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:15.103830 30734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:15.105161 30734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:15.114568 30740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:15.114609 30739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:15.114583 30742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:15.114791 30734 server_base.cc:1034] running on GCE node
I20260926 16:41:15.115236 30734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:15.115826 30734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:15.117017 30734 hybrid_clock.cc:648] HybridClock initialized: now 1790440875117005 us; error 26 us; skew 500 ppm
I20260926 16:41:15.118772 30734 webserver.cc:460] Webserver started at http://127.11.158.130:38515/ using document root <none> and password file <none>
I20260926 16:41:15.119351 30734 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:15.119444 30734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:15.119678 30734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:15.121881 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "5918f698133743c5a07907ca231615bb"
format_stamp: "Formatted at 2026-09-26 16:41:15 on dist-test-slave-p39x"
server_key: "0faba8062f738e36cdc375006f467ee5"
server_key_iv: "ea7391133b13b1bf82454af0889c50f1"
server_key_version: "encryptionkey@0"
I20260926 16:41:15.122462 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "5918f698133743c5a07907ca231615bb"
format_stamp: "Formatted at 2026-09-26 16:41:15 on dist-test-slave-p39x"
server_key: "0faba8062f738e36cdc375006f467ee5"
server_key_iv: "ea7391133b13b1bf82454af0889c50f1"
server_key_version: "encryptionkey@0"
I20260926 16:41:15.126153 30734 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:41:15.128587 30748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:15.129801 30734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:15.129907 30734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "5918f698133743c5a07907ca231615bb"
format_stamp: "Formatted at 2026-09-26 16:41:15 on dist-test-slave-p39x"
server_key: "0faba8062f738e36cdc375006f467ee5"
server_key_iv: "ea7391133b13b1bf82454af0889c50f1"
server_key_version: "encryptionkey@0"
I20260926 16:41:15.129992 30734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:15.144131 30734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:15.144815 30734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:15.144965 30734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:15.145479 30734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:15.146353 30734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:15.146397 30734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:15.146436 30734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:15.146452 30734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:15.155781 30734 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:34037
I20260926 16:41:15.155820 30861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:34037 every 8 connection(s)
I20260926 16:41:15.156749 30734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:41:15.158924 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 30734
I20260926 16:41:15.159046 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:41:15.159417 11898 external_mini_cluster.cc:1428] Setting key 2581822c0559a41ce7e95f2a456c54cf
I20260926 16:41:15.161576 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38615
--builtin_ntp_servers=127.11.158.148:40229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:41:15.167614 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.158591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:36059 (local address 127.11.158.190:38615)
0926 16:41:15.158744 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.158748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.160150 (+ 1402us) server_negotiation.cc:407] Connection header received
0926 16:41:15.161920 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.161923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.161997 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.162114 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:15.163283 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.163809 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.165632 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.165867 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.165982 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.167104 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:15.167108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:15.167111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:15.167131 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:15.167150 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:15.167158 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:15.167239 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.167406 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.167469 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 16:41:15.168566 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.159093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38615 (local address 127.11.158.130:36059)
0926 16:41:15.159980 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.160041 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:15.161655 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:15.162303 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:15.162310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:15.162400 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:15.163111 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.163134 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.164481 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.164484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.165498 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.165508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.166022 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.166026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.166197 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.166835 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:15.166837 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:15.166877 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:15.166940 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:15.167241 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:15.167254 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:15.167260 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:41:15.167405 (+ 145us) client_negotiation.cc:240] Negotiation successful
0926 16:41:15.167689 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":146,"threads_started":1}
I20260926 16:41:15.169976 30862 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38615
I20260926 16:41:15.170215 30862 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:15.170750 30862 heartbeater.cc:511] Master 127.11.158.190:38615 requested a full tablet report, sending...
I20260926 16:41:15.171926 30543 ts_manager.cc:194] Registered new tserver with Master: 5918f698133743c5a07907ca231615bb (127.11.158.130:34037)
I20260926 16:41:15.172933 30543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.130:36059
W20260926 16:41:15.279325 30865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:15.279585 30865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:15.279623 30865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:15.283593 30865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:15.283650 30865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:15.283705 30865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:41:15.288411 30865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:40229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:15.289395 30865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:15.290704 30865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:15.300189 30873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:15.300173 30870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:15.300200 30871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:15.300459 30865 server_base.cc:1034] running on GCE node
I20260926 16:41:15.300923 30865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:15.301555 30865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:15.302734 30865 hybrid_clock.cc:648] HybridClock initialized: now 1790440875302700 us; error 53 us; skew 500 ppm
I20260926 16:41:15.304499 30865 webserver.cc:460] Webserver started at http://127.11.158.131:42105/ using document root <none> and password file <none>
I20260926 16:41:15.305066 30865 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:15.305154 30865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:15.305384 30865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:15.307374 30865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca"
format_stamp: "Formatted at 2026-09-26 16:41:15 on dist-test-slave-p39x"
server_key: "4124947b3d4bb6c67a5706d7465da74e"
server_key_iv: "4352cff747bab306875f0b8f62beef90"
server_key_version: "encryptionkey@0"
I20260926 16:41:15.307931 30865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca"
format_stamp: "Formatted at 2026-09-26 16:41:15 on dist-test-slave-p39x"
server_key: "4124947b3d4bb6c67a5706d7465da74e"
server_key_iv: "4352cff747bab306875f0b8f62beef90"
server_key_version: "encryptionkey@0"
I20260926 16:41:15.311499 30865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:41:15.313951 30879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:15.315171 30865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:41:15.315301 30865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca"
format_stamp: "Formatted at 2026-09-26 16:41:15 on dist-test-slave-p39x"
server_key: "4124947b3d4bb6c67a5706d7465da74e"
server_key_iv: "4352cff747bab306875f0b8f62beef90"
server_key_version: "encryptionkey@0"
I20260926 16:41:15.315407 30865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:15.333614 30865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:15.334321 30865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:15.334501 30865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:15.335032 30865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:15.335932 30865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:15.335979 30865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:15.336018 30865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:15.336041 30865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:15.345074 30865 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:43283
I20260926 16:41:15.345100 30992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:43283 every 8 connection(s)
I20260926 16:41:15.345973 30865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:41:15.348026 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 30865
I20260926 16:41:15.348167 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:41:15.348524 11898 external_mini_cluster.cc:1428] Setting key 6b0ebe5117619cec507d2cfd6c778d64
I20260926 16:41:15.355401 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.347810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:53677 (local address 127.11.158.190:38615)
0926 16:41:15.347970 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.347975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.349301 (+ 1326us) server_negotiation.cc:407] Connection header received
0926 16:41:15.350882 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.350885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.350938 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.351017 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:15.351871 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.352490 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.353602 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.353842 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.353953 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.354919 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:15.354922 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:15.354925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:15.354938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:15.354966 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:15.354974 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:15.355051 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.355205 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.355265 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:41:15.356213 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.348294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38615 (local address 127.11.158.131:53677)
0926 16:41:15.349135 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.349190 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:15.350689 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:15.351182 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:15.351189 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:15.351273 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:15.351736 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.351749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.352619 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.352622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.353480 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.353489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.353996 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.354000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.354157 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.354677 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:15.354679 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:15.354715 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:15.354775 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:15.355064 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:15.355076 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:15.355085 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:41:15.355205 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 16:41:15.355468 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":133,"threads_started":1}
I20260926 16:41:15.357268 30993 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38615
I20260926 16:41:15.357509 30993 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:15.357983 30993 heartbeater.cc:511] Master 127.11.158.190:38615 requested a full tablet report, sending...
I20260926 16:41:15.358942 30543 ts_manager.cc:194] Registered new tserver with Master: bd1a2ba6b6614bf680b45eeb2dcc0fca (127.11.158.131:43283)
I20260926 16:41:15.359519 30543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.131:53677
I20260926 16:41:15.362283 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:41:15.367990 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.364193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.11.158.190:38615)
0926 16:41:15.364325 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.364329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.364558 (+ 229us) server_negotiation.cc:407] Connection header received
0926 16:41:15.364690 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.364693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.364733 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.364809 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:15.365074 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.365582 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.366425 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.366654 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.366749 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.367549 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:15.367552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:15.367554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:15.367566 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:15.367585 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:15.367593 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:15.367701 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.367845 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.367880 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 16:41:15.371121 30543 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:41:15.374191 30543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:41:15.388290 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.382644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48210 (local address 127.11.158.130:34037)
0926 16:41:15.383081 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.383095 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.383141 (+ 46us) server_negotiation.cc:407] Connection header received
0926 16:41:15.383223 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.383228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.383447 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.383655 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:15.383972 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.385190 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.386717 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.387538 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.387674 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.387859 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.388025 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.388123 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":127,"threads_started":1}
I20260926 16:41:15.388299 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.382816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:42513 (local address 127.0.0.1:49926)
0926 16:41:15.383254 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.383266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:15.383340 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:15.384105 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:15.384108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:15.384126 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:15.384208 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.384213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.385602 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.385607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.386614 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.386624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.387694 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.387697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.387835 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.387860 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:41:15.387963 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 16:41:15.388097 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":25,"thread_start_us":111,"threads_started":1}
I20260926 16:41:15.388285 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.382467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:34037 (local address 127.0.0.1:48210)
0926 16:41:15.382855 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.382909 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:15.383031 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:15.383663 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:15.383669 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:15.383695 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:15.383825 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.383831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.385309 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.385312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.386585 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.386593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.387658 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.387662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.387811 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.387854 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 16:41:15.387995 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 16:41:15.388104 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":129,"thread_start_us":100,"threads_started":1}
I20260926 16:41:15.388937 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.383219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:43283 (local address 127.0.0.1:35892)
0926 16:41:15.384011 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.384027 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:15.384108 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:15.384799 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:15.384802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:15.384819 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:15.384915 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.384924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.386234 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.386237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.387303 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.387312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.388574 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.388577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.388674 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.388695 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:41:15.388767 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 16:41:15.388827 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":72,"threads_started":1}
I20260926 16:41:15.389281 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.383267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.11.158.129:42513)
0926 16:41:15.383668 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.383679 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.383703 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:41:15.383772 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.383777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.383961 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.384364 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:15.384501 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.385493 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.386755 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.387564 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.388780 (+ 1216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.388952 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.389007 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.389118 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260926 16:41:15.389454 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.383607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35892 (local address 127.11.158.131:43283)
0926 16:41:15.384363 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.384383 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.384407 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:41:15.384479 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.384483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.384669 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.384819 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:15.385040 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.386107 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.387452 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.388198 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.388310 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.388509 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.389183 (+ 674us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.389276 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":146,"threads_started":1}
I20260926 16:41:15.391441 30796 tablet_service.cc:1467] Processing CreateTablet for tablet efdfcbabbdd34bd5bebca34b137db5c9 (DEFAULT_TABLE table=test-table [id=6d92716061624ae990738cc8a5d5ab26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:15.391867 30665 tablet_service.cc:1467] Processing CreateTablet for tablet efdfcbabbdd34bd5bebca34b137db5c9 (DEFAULT_TABLE table=test-table [id=6d92716061624ae990738cc8a5d5ab26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:15.391867 30927 tablet_service.cc:1467] Processing CreateTablet for tablet efdfcbabbdd34bd5bebca34b137db5c9 (DEFAULT_TABLE table=test-table [id=6d92716061624ae990738cc8a5d5ab26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:15.392498 30796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdfcbabbdd34bd5bebca34b137db5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:15.392918 30665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdfcbabbdd34bd5bebca34b137db5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:15.392918 30927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdfcbabbdd34bd5bebca34b137db5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:15.399740 31010 tablet_bootstrap.cc:492] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Bootstrap starting.
I20260926 16:41:15.400744 31012 tablet_bootstrap.cc:492] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22: Bootstrap starting.
I20260926 16:41:15.402544 31010 tablet_bootstrap.cc:654] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:15.402580 31012 tablet_bootstrap.cc:654] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:15.403286 31012 log.cc:824] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:15.403286 31010 log.cc:824] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:15.403748 31011 tablet_bootstrap.cc:492] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Bootstrap starting.
I20260926 16:41:15.405246 31010 tablet_bootstrap.cc:492] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: No bootstrap required, opened a new log
I20260926 16:41:15.405465 31010 ts_tablet_manager.cc:1397] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:41:15.405896 31012 tablet_bootstrap.cc:492] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22: No bootstrap required, opened a new log
I20260926 16:41:15.406002 31011 tablet_bootstrap.cc:654] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:15.406090 31012 ts_tablet_manager.cc:1397] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260926 16:41:15.406832 31011 log.cc:824] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:15.409027 31011 tablet_bootstrap.cc:492] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: No bootstrap required, opened a new log
I20260926 16:41:15.409212 31011 ts_tablet_manager.cc:1397] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260926 16:41:15.409458 31010 raft_consensus.cc:348] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.409705 31010 raft_consensus.cc:374] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:15.409756 31010 raft_consensus.cc:729] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5918f698133743c5a07907ca231615bb, State: Initialized, Role: FOLLOWER
I20260926 16:41:15.409924 31012 raft_consensus.cc:348] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.410169 31012 raft_consensus.cc:374] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:15.410225 31012 raft_consensus.cc:729] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cefe510553c4e0bbdfce46b182e3e22, State: Initialized, Role: FOLLOWER
I20260926 16:41:15.410243 31010 consensus_queue.cc:260] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.410632 31012 consensus_queue.cc:260] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.411005 30862 heartbeater.cc:503] Master 127.11.158.190:38615 was elected leader, sending a full tablet report...
I20260926 16:41:15.411329 31010 ts_tablet_manager.cc:1428] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:41:15.411589 31012 ts_tablet_manager.cc:1428] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260926 16:41:15.411919 30731 heartbeater.cc:503] Master 127.11.158.190:38615 was elected leader, sending a full tablet report...
I20260926 16:41:15.413697 31011 raft_consensus.cc:348] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.413995 31011 raft_consensus.cc:374] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:15.414063 31011 raft_consensus.cc:729] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd1a2ba6b6614bf680b45eeb2dcc0fca, State: Initialized, Role: FOLLOWER
I20260926 16:41:15.414573 31011 consensus_queue.cc:260] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.415320 30993 heartbeater.cc:503] Master 127.11.158.190:38615 was elected leader, sending a full tablet report...
I20260926 16:41:15.415558 31011 ts_tablet_manager.cc:1428] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260926 16:41:15.447237 31017 raft_consensus.cc:482] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:41:15.447475 31017 raft_consensus.cc:504] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.448778 31017 leader_election.cc:288] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5918f698133743c5a07907ca231615bb (127.11.158.130:34037), bd1a2ba6b6614bf680b45eeb2dcc0fca (127.11.158.131:43283)
I20260926 16:41:15.453128 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.449133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:34037 (local address 127.11.158.129:53957)
0926 16:41:15.449271 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.449287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:15.449393 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:15.449870 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:15.449874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:15.449892 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:15.450017 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.450023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.451062 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.451065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.451882 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.451891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.452706 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.452710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.452801 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.452828 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:41:15.452908 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:41:15.452988 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 16:41:15.453143 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.449212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:53957 (local address 127.11.158.130:34037)
0926 16:41:15.449350 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.449354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.449370 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:41:15.449687 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.449693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.449752 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.449845 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:15.450141 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.450947 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.452011 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.452587 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.452710 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.452749 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.452930 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.452996 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:41:15.453516 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.449553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:50521 (local address 127.11.158.131:43283)
0926 16:41:15.449675 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.449679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.449819 (+ 140us) server_negotiation.cc:407] Connection header received
0926 16:41:15.449961 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.449966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.450009 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.450072 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:15.450326 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.451031 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.452310 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.452957 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.453063 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.453103 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.453337 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.453408 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:41:15.453558 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.449434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:43283 (local address 127.11.158.129:50521)
0926 16:41:15.449736 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.449747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:15.449857 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:15.450106 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:15.450110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:15.450129 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:15.450228 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.450231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.451118 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.451123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.452189 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:15.452198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.453074 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.453077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:15.453175 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.453200 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:41:15.453319 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 16:41:15.453403 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":115,"threads_started":1}
I20260926 16:41:15.453841 30816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdfcbabbdd34bd5bebca34b137db5c9" candidate_uuid: "0cefe510553c4e0bbdfce46b182e3e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5918f698133743c5a07907ca231615bb" is_pre_election: true
I20260926 16:41:15.454154 30816 raft_consensus.cc:2454] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cefe510553c4e0bbdfce46b182e3e22 in term 0.
I20260926 16:41:15.454613 30619 leader_election.cc:302] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cefe510553c4e0bbdfce46b182e3e22, 5918f698133743c5a07907ca231615bb; no voters:
I20260926 16:41:15.454876 30947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdfcbabbdd34bd5bebca34b137db5c9" candidate_uuid: "0cefe510553c4e0bbdfce46b182e3e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" is_pre_election: true
I20260926 16:41:15.454913 31017 raft_consensus.cc:2781] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:41:15.455013 31017 raft_consensus.cc:482] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:41:15.455075 31017 raft_consensus.cc:3037] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:15.455168 30947 raft_consensus.cc:2454] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cefe510553c4e0bbdfce46b182e3e22 in term 0.
I20260926 16:41:15.456246 31017 raft_consensus.cc:504] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.456630 31017 leader_election.cc:288] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [CANDIDATE]: Term 1 election: Requested vote from peers 5918f698133743c5a07907ca231615bb (127.11.158.130:34037), bd1a2ba6b6614bf680b45eeb2dcc0fca (127.11.158.131:43283)
I20260926 16:41:15.457005 30947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdfcbabbdd34bd5bebca34b137db5c9" candidate_uuid: "0cefe510553c4e0bbdfce46b182e3e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca"
I20260926 16:41:15.456997 30816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdfcbabbdd34bd5bebca34b137db5c9" candidate_uuid: "0cefe510553c4e0bbdfce46b182e3e22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5918f698133743c5a07907ca231615bb"
I20260926 16:41:15.457119 30816 raft_consensus.cc:3037] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:15.457118 30947 raft_consensus.cc:3037] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:15.458359 30816 raft_consensus.cc:2454] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cefe510553c4e0bbdfce46b182e3e22 in term 1.
I20260926 16:41:15.458359 30947 raft_consensus.cc:2454] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cefe510553c4e0bbdfce46b182e3e22 in term 1.
I20260926 16:41:15.458652 30619 leader_election.cc:302] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cefe510553c4e0bbdfce46b182e3e22, 5918f698133743c5a07907ca231615bb; no voters:
I20260926 16:41:15.458806 31017 raft_consensus.cc:2781] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:15.459028 31017 raft_consensus.cc:686] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [term 1 LEADER]: Becoming Leader. State: Replica: 0cefe510553c4e0bbdfce46b182e3e22, State: Running, Role: LEADER
I20260926 16:41:15.459358 31017 consensus_queue.cc:237] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } }
I20260926 16:41:15.462898 30542 catalog_manager.cc:5495] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cefe510553c4e0bbdfce46b182e3e22 (127.11.158.129). New cstate: current_term: 1 leader_uuid: "0cefe510553c4e0bbdfce46b182e3e22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cefe510553c4e0bbdfce46b182e3e22" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 42513 } health_report { overall_health: HEALTHY } } }
I20260926 16:41:15.476187 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.472483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.11.158.129:42513)
0926 16:41:15.472619 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:15.472623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:15.472636 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:41:15.472704 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:15.472706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:15.472759 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:15.472850 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:15.473091 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.473614 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.474483 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:15.474696 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:15.474778 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:15.475106 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:15.475553 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:15.475713 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:15.475955 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 16:41:15.476000 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 16:41:15.477094 30732 tablet.cc:2369] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:41:15.481967 30816 raft_consensus.cc:1261] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 1 FOLLOWER]: Refusing update from remote peer 0cefe510553c4e0bbdfce46b182e3e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:15.481971 30947 raft_consensus.cc:1261] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 1 FOLLOWER]: Refusing update from remote peer 0cefe510553c4e0bbdfce46b182e3e22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:15.482676 31017 consensus_queue.cc:1035] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5918f698133743c5a07907ca231615bb" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:15.482843 31020 consensus_queue.cc:1035] T efdfcbabbdd34bd5bebca34b137db5c9 P 0cefe510553c4e0bbdfce46b182e3e22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd1a2ba6b6614bf680b45eeb2dcc0fca" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 43283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:15.490722 31025 mvcc.cc:205] Tried to move back new op lower bound from 7333645825966751744 to 7333645825884504064. Current Snapshot: MvccSnapshot[applied={T|T < 7333645825966751744}]
I20260926 16:41:15.499526 30543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:15.499708 30543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:15.502184 30543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2797a742daf34e59bc2ff6366ca717de: Sending DeleteTablet for 3 replicas of tablet efdfcbabbdd34bd5bebca34b137db5c9
I20260926 16:41:15.503109 30796 tablet_service.cc:1514] Processing DeleteTablet for tablet efdfcbabbdd34bd5bebca34b137db5c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:15 UTC) from {username='slave'} at 127.0.0.1:48210
I20260926 16:41:15.503134 30927 tablet_service.cc:1514] Processing DeleteTablet for tablet efdfcbabbdd34bd5bebca34b137db5c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:15 UTC) from {username='slave'} at 127.0.0.1:35892
I20260926 16:41:15.503582 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 30603
I20260926 16:41:15.503602 30665 tablet_service.cc:1514] Processing DeleteTablet for tablet efdfcbabbdd34bd5bebca34b137db5c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:15 UTC) from {username='slave'} at 127.0.0.1:49926
I20260926 16:41:15.504621 31034 tablet_replica.cc:330] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: stopping tablet replica
I20260926 16:41:15.504856 31033 tablet_replica.cc:330] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: stopping tablet replica
I20260926 16:41:15.505182 31033 raft_consensus.cc:2229] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:15.505486 31033 raft_consensus.cc:2258] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:15.505959 31034 raft_consensus.cc:2229] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:15.506244 31034 raft_consensus.cc:2258] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:15.507033 31033 ts_tablet_manager.cc:1905] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:15.507555 31034 ts_tablet_manager.cc:1905] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:15.509788 31033 ts_tablet_manager.cc:1918] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:15.509888 31033 log.cc:1196] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/efdfcbabbdd34bd5bebca34b137db5c9
I20260926 16:41:15.510138 31034 ts_tablet_manager.cc:1918] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:15.510211 31033 ts_tablet_manager.cc:1939] T efdfcbabbdd34bd5bebca34b137db5c9 P 5918f698133743c5a07907ca231615bb: Deleting consensus metadata
I20260926 16:41:15.510228 31034 log.cc:1196] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/efdfcbabbdd34bd5bebca34b137db5c9
I20260926 16:41:15.510512 31034 ts_tablet_manager.cc:1939] T efdfcbabbdd34bd5bebca34b137db5c9 P bd1a2ba6b6614bf680b45eeb2dcc0fca: Deleting consensus metadata
W20260926 16:41:15.510531 30529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:42513 (error 108)
I20260926 16:41:15.510823 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 30734
I20260926 16:41:15.511582 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.511262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:42513 (local address 127.0.0.1:49946)
0926 16:41:15.511430 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.511496 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:42513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260926 16:41:15.511740 30529 catalog_manager.cc:4584] TS 0cefe510553c4e0bbdfce46b182e3e22 (127.11.158.129:42513): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet efdfcbabbdd34bd5bebca34b137db5c9: Network error: Client connection negotiation failed: client connection to 127.11.158.129:42513: connect: Connection refused (error 111)
I20260926 16:41:15.511795 30528 catalog_manager.cc:4841] TS bd1a2ba6b6614bf680b45eeb2dcc0fca (127.11.158.131:43283): tablet efdfcbabbdd34bd5bebca34b137db5c9 (table test-table [id=6d92716061624ae990738cc8a5d5ab26]) successfully deleted
I20260926 16:41:15.516677 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 30865
I20260926 16:41:15.517162 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:15.516849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:34037 (local address 127.0.0.1:48226)
0926 16:41:15.517024 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:15.517076 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.130:34037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 16:41:15.522783 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 30512
2026-09-26T16:41:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1006 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:15 dist-test-slave-p39x krb5kdc[31038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:15 dist-test-slave-p39x krb5kdc[31038](info): set up 2 sockets
Sep 26 16:41:15 dist-test-slave-p39x krb5kdc[31038](info): commencing operation
krb5kdc: starting...
W20260926 16:41:16.561074 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:16 dist-test-slave-p39x krb5kdc[31038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:16.688732 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:37617
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:37617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:16.801411 31051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:16.801740 31051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:16.801821 31051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:16.805922 31051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:16.806023 31051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:16.806062 31051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:16.806113 31051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:16.806161 31051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:16.811558 31051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:37617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:37617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:16.812717 31051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:16.813751 31051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:16.822494 31051 server_base.cc:1034] running on GCE node
W20260926 16:41:16.822548 31059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:16.822571 31057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:16.822645 31056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:16.823347 31051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:16.824527 31051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:16.825712 31051 hybrid_clock.cc:648] HybridClock initialized: now 1790440876825690 us; error 36 us; skew 500 ppm
Sep 26 16:41:16 dist-test-slave-p39x krb5kdc[31038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440876, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:16.828547 31051 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:16.829494 31051 webserver.cc:460] Webserver started at http://127.11.158.190:36603/ using document root <none> and password file <none>
I20260926 16:41:16.830061 31051 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:16.830153 31051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:16.830365 31051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:16.832546 31051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "b7be3fcfdb7a4d9d94b006f73736b766"
format_stamp: "Formatted at 2026-09-26 16:41:16 on dist-test-slave-p39x"
server_key: "1b7d067610378f506ae0d728dce03f75"
server_key_iv: "af45fa1f1527a9bb76a90dde90af4e87"
server_key_version: "encryptionkey@0"
I20260926 16:41:16.833097 31051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "b7be3fcfdb7a4d9d94b006f73736b766"
format_stamp: "Formatted at 2026-09-26 16:41:16 on dist-test-slave-p39x"
server_key: "1b7d067610378f506ae0d728dce03f75"
server_key_iv: "af45fa1f1527a9bb76a90dde90af4e87"
server_key_version: "encryptionkey@0"
I20260926 16:41:16.836752 31051 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:41:16.839154 31066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:16.840384 31051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:16.840543 31051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "b7be3fcfdb7a4d9d94b006f73736b766"
format_stamp: "Formatted at 2026-09-26 16:41:16 on dist-test-slave-p39x"
server_key: "1b7d067610378f506ae0d728dce03f75"
server_key_iv: "af45fa1f1527a9bb76a90dde90af4e87"
server_key_version: "encryptionkey@0"
I20260926 16:41:16.840656 31051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:16.856132 31051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:16.859354 31051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:16.859576 31051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:16.866947 31051 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:37617
I20260926 16:41:16.866976 31118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:37617 every 8 connection(s)
I20260926 16:41:16.867923 31051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:16.870689 31119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:16.875443 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31051
I20260926 16:41:16.875602 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:16.875906 11898 external_mini_cluster.cc:1428] Setting key 31572c5c3a1da57a40cafd02f6ca155f
I20260926 16:41:16.877023 31119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766: Bootstrap starting.
I20260926 16:41:16.879577 31119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:16.880451 31119 log.cc:824] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:16.882357 31119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766: No bootstrap required, opened a new log
Sep 26 16:41:16 dist-test-slave-p39x krb5kdc[31038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:16.886967 31119 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 37617 } }
I20260926 16:41:16.887284 31119 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:16.887411 31119 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7be3fcfdb7a4d9d94b006f73736b766, State: Initialized, Role: FOLLOWER
I20260926 16:41:16.887966 31119 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 37617 } }
I20260926 16:41:16.888141 31119 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:16.888216 31119 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:16.888324 31119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:16.889569 31119 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 37617 } }
I20260926 16:41:16.889953 31119 leader_election.cc:302] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7be3fcfdb7a4d9d94b006f73736b766; no voters:
I20260926 16:41:16.890451 31119 leader_election.cc:288] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:16.891695 31124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:16.891829 31119 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:16.891958 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:16.877564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.11.158.190:37617)
0926 16:41:16.878032 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:16.878047 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:16.878083 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:41:16.878875 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:16.878903 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:16.879244 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:16.879595 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:16.880433 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:16.881665 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:16.882591 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:16.883261 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:16.883437 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:16.886217 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:16.886223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:16.886242 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:16.886273 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:16.888446 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:16.888883 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:16.888890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:16.888896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:16.889037 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:16.889304 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:16.889307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:16.889309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:16.889871 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:16.890076 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:16.890616 (+ 540us) server_negotiation.cc:299] Negotiation successful
0926 16:41:16.891005 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":142,"threads_started":1}
I20260926 16:41:16.892004 31124 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [term 1 LEADER]: Becoming Leader. State: Replica: b7be3fcfdb7a4d9d94b006f73736b766, State: Running, Role: LEADER
I20260926 16:41:16.892571 31124 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 37617 } }
I20260926 16:41:16.893877 31124 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7be3fcfdb7a4d9d94b006f73736b766. Latest consensus state: current_term: 1 leader_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 37617 } } }
I20260926 16:41:16.894007 31124 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:16.894623 31126 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7be3fcfdb7a4d9d94b006f73736b766" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 37617 } } }
I20260926 16:41:16.895933 31126 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766 [sys.catalog]: This master's current role is: LEADER
W20260926 16:41:16.897143 31138 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:16.897228 31138 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:16.897356 31139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:16.898258 31139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:16.903908 31139 catalog_manager.cc:1347] Generated new cluster ID: 8274b508da674e2a859c8819133db504
I20260926 16:41:16.903999 31139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:16.925840 31139 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:16.927186 31139 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:16.937781 31139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766: Generated new TSK 0
I20260926 16:41:16.938681 31139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.158.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
Entry for principal kudu/127.11.158.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab.
I20260926 16:41:16.972638 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:37617
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:17.087221 31145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:17.087568 31145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:17.087651 31145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:17.091877 31145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:17.091984 31145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:17.092098 31145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:41:17.097733 31145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.129.keytab
--principal=kudu/127.11.158.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:37617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:17.098920 31145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:17.099998 31145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:17.110155 31150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:17.110246 31151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:17.110219 31153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:17.110211 31145 server_base.cc:1034] running on GCE node
I20260926 16:41:17.110939 31145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:17.111665 31145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:17.112856 31145 hybrid_clock.cc:648] HybridClock initialized: now 1790440877112840 us; error 40 us; skew 500 ppm
Sep 26 16:41:17 dist-test-slave-p39x krb5kdc[31038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440877, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:17.115743 31145 init.cc:377] Logged in from keytab as kudu/127.11.158.129@KRBTEST.COM (short username kudu)
I20260926 16:41:17.116796 31145 webserver.cc:460] Webserver started at http://127.11.158.129:40627/ using document root <none> and password file <none>
I20260926 16:41:17.117389 31145 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:17.117484 31145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:17.117728 31145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:17.119916 31145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "f846f400fdd54285a37bb5ea5e0caadf"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "a2a3bb768f37ae26b856abcc667bb0c7"
server_key_iv: "35fa66b83f8d539b9fa0322d500f0444"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.120568 31145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "f846f400fdd54285a37bb5ea5e0caadf"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "a2a3bb768f37ae26b856abcc667bb0c7"
server_key_iv: "35fa66b83f8d539b9fa0322d500f0444"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.124516 31145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:41:17.126996 31160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.128225 31145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:41:17.128408 31145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "f846f400fdd54285a37bb5ea5e0caadf"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "a2a3bb768f37ae26b856abcc667bb0c7"
server_key_iv: "35fa66b83f8d539b9fa0322d500f0444"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.128542 31145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:17.147691 31145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:17.151155 31145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:17.151386 31145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:17.152004 31145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:17.152891 31145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:17.152938 31145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.153012 31145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:17.153043 31145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.161844 31145 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:40557
I20260926 16:41:17.161862 31273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:40557 every 8 connection(s)
I20260926 16:41:17.162750 31145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:41:17.169962 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31145
I20260926 16:41:17.170120 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:41:17.170408 11898 external_mini_cluster.cc:1428] Setting key 8889915ca51d840c927c81e64c519aed
Sep 26 16:41:17 dist-test-slave-p39x krb5kdc[31038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440877, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.129@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:17.176954 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.164670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:44413 (local address 127.11.158.190:37617)
0926 16:41:17.164826 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.164831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.165746 (+ 915us) server_negotiation.cc:407] Connection header received
0926 16:41:17.167027 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.167029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.167078 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.167154 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:17.168311 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.168883 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.170089 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.170414 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.170537 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.173111 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:17.173119 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:17.173121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:17.173146 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:17.174688 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.175234 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.175239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.175241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.175327 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.175697 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.175702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.175704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.175908 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:17.176021 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.176619 (+ 598us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.176763 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:41:17.177781 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.165136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:37617 (local address 127.11.158.129:44413)
0926 16:41:17.165591 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.165634 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.166820 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.167316 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.167323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.167699 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:17.168182 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.168195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.169013 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.169016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.169944 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.169954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.170587 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.170592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.170760 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.171397 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:17.171413 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:17.172934 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:17.174819 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:17.174825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:17.174857 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:17.175123 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:17.175431 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:17.175434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:17.175436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:17.175598 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:17.176020 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:17.176027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:17.176297 (+ 270us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.176566 (+ 269us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.176917 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260926 16:41:17.179158 31274 heartbeater.cc:347] Connected to a master server at 127.11.158.190:37617
I20260926 16:41:17.179450 31274 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.158.130@KRBTEST.COM; defaulting to no policy
I20260926 16:41:17.180033 31274 heartbeater.cc:511] Master 127.11.158.190:37617 requested a full tablet report, sending...
I20260926 16:41:17.181738 31083 ts_manager.cc:194] Registered new tserver with Master: f846f400fdd54285a37bb5ea5e0caadf (127.11.158.129:40557)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.130@KRBTEST.COM" created.
I20260926 16:41:17.183197 31083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.129@KRBTEST.COM'} at 127.11.158.129:44413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
Entry for principal kudu/127.11.158.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab.
I20260926 16:41:17.199764 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:37617
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:17.314148 31279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:17.314421 31279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:17.314466 31279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:17.318559 31279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:17.318624 31279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:17.318682 31279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:41:17.323987 31279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.130.keytab
--principal=kudu/127.11.158.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:37617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:17.325075 31279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:17.326138 31279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:17.336292 31279 server_base.cc:1034] running on GCE node
W20260926 16:41:17.336254 31284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:17.336248 31285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:17.336287 31287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:17.336902 31279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:17.337514 31279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:17.338675 31279 hybrid_clock.cc:648] HybridClock initialized: now 1790440877338652 us; error 35 us; skew 500 ppm
Sep 26 16:41:17 dist-test-slave-p39x krb5kdc[31038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440877, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:17.341425 31279 init.cc:377] Logged in from keytab as kudu/127.11.158.130@KRBTEST.COM (short username kudu)
I20260926 16:41:17.342356 31279 webserver.cc:460] Webserver started at http://127.11.158.130:42723/ using document root <none> and password file <none>
I20260926 16:41:17.342934 31279 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:17.342998 31279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:17.343261 31279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:17.345393 31279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "311d5af88aff3ff97fbe6358e584b617"
server_key_iv: "c035a43e2cd2b301950e86facd5d8122"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.345986 31279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "311d5af88aff3ff97fbe6358e584b617"
server_key_iv: "c035a43e2cd2b301950e86facd5d8122"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.349620 31279 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:41:17.351976 31294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.353307 31279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:41:17.353457 31279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "311d5af88aff3ff97fbe6358e584b617"
server_key_iv: "c035a43e2cd2b301950e86facd5d8122"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.353572 31279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:17.371654 31279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:17.374501 31279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:17.374702 31279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:17.375288 31279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:17.376102 31279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:17.376173 31279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.376238 31279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:17.376291 31279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.385711 31279 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:37915
I20260926 16:41:17.385737 31407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:37915 every 8 connection(s)
I20260926 16:41:17.386586 31279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
Sep 26 16:41:17 dist-test-slave-p39x krb5kdc[31038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440877, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.130@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:17.396602 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31279
I20260926 16:41:17.396730 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:41:17.397006 11898 external_mini_cluster.cc:1428] Setting key 1b3770d2a0d515d355944972cfae9c3d
I20260926 16:41:17.401068 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.388324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:37363 (local address 127.11.158.190:37617)
0926 16:41:17.388485 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.388489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.389334 (+ 845us) server_negotiation.cc:407] Connection header received
0926 16:41:17.390615 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.390618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.390672 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.390779 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:17.391887 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.392573 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.393862 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.394206 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.394323 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.396847 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:17.396851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:17.396853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:17.396878 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:17.398629 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.399267 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.399269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.399271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.399359 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.399744 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.399746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.399747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.400011 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:17.400128 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.400688 (+ 560us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.400853 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:41:17.401760 31410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.388759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:37617 (local address 127.11.158.130:37363)
0926 16:41:17.389191 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.389232 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.390401 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.390945 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.390952 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.391286 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:17.391747 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.391760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.392698 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.392706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.393727 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.393737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.394362 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.394367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.394544 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.395074 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:17.395090 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:17.396646 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:17.398741 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:17.398747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:17.398764 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:17.399159 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:17.399453 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:17.399458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:17.399463 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:17.399648 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:17.400138 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:17.400144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:17.400448 (+ 304us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.400610 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.400865 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260926 16:41:17.402976 31408 heartbeater.cc:347] Connected to a master server at 127.11.158.190:37617
I20260926 16:41:17.403237 31408 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:17.403872 31408 heartbeater.cc:511] Master 127.11.158.190:37617 requested a full tablet report, sending...
I20260926 16:41:17.405031 31083 ts_manager.cc:194] Registered new tserver with Master: 720abcffdcee4c86bd9bbaaf7da8c9ba (127.11.158.130:37915)
I20260926 16:41:17.405608 31083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.130@KRBTEST.COM'} at 127.11.158.130:37363
WARNING: no policy specified for kudu/127.11.158.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
Entry for principal kudu/127.11.158.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab.
I20260926 16:41:17.425741 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:37617
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:17.537549 31413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:17.537886 31413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:17.537974 31413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:17.542264 31413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:17.542371 31413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:17.542465 31413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:41:17.548187 31413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.131.keytab
--principal=kudu/127.11.158.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:37617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:17.549396 31413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:17.550457 31413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:17.560529 31418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:17.560518 31419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:17.560600 31413 server_base.cc:1034] running on GCE node
W20260926 16:41:17.560781 31421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:17.561365 31413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:17.562130 31413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:17.563318 31413 hybrid_clock.cc:648] HybridClock initialized: now 1790440877563304 us; error 26 us; skew 500 ppm
Sep 26 16:41:17 dist-test-slave-p39x krb5kdc[31038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440877, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:17.566375 31413 init.cc:377] Logged in from keytab as kudu/127.11.158.131@KRBTEST.COM (short username kudu)
I20260926 16:41:17.567322 31413 webserver.cc:460] Webserver started at http://127.11.158.131:44795/ using document root <none> and password file <none>
I20260926 16:41:17.567950 31413 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:17.568048 31413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:17.568288 31413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:17.570439 31413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "ea49997c1789466c8af351df0692087a"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "6bdc4312b867d28a187d6ea7648d1a27"
server_key_iv: "d5bd05c8570eaf448baa737737e1964e"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.571043 31413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "ea49997c1789466c8af351df0692087a"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "6bdc4312b867d28a187d6ea7648d1a27"
server_key_iv: "d5bd05c8570eaf448baa737737e1964e"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.575052 31413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 16:41:17.577761 31428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.579075 31413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:41:17.579231 31413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "ea49997c1789466c8af351df0692087a"
format_stamp: "Formatted at 2026-09-26 16:41:17 on dist-test-slave-p39x"
server_key: "6bdc4312b867d28a187d6ea7648d1a27"
server_key_iv: "d5bd05c8570eaf448baa737737e1964e"
server_key_version: "encryptionkey@0"
I20260926 16:41:17.579363 31413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:17.590131 31413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:17.594259 31413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:17.594484 31413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:17.595125 31413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:17.595992 31413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:17.596066 31413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.596131 31413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:17.596161 31413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:17.605455 31413 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:35653
I20260926 16:41:17.605489 31541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:35653 every 8 connection(s)
I20260926 16:41:17.606379 31413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:41:17.612510 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31413
I20260926 16:41:17.612644 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:41:17.612936 11898 external_mini_cluster.cc:1428] Setting key 41f66938924df8a03257448d4ea7300d
Sep 26 16:41:17 dist-test-slave-p39x krb5kdc[31038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440877, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.131@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:17.620635 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.608293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:34395 (local address 127.11.158.190:37617)
0926 16:41:17.608508 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.608512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.609358 (+ 846us) server_negotiation.cc:407] Connection header received
0926 16:41:17.610710 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.610714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.610775 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.610883 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:17.612116 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.612819 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.614031 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.614275 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.614387 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.616931 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:17.616935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:17.616944 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:17.616978 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:17.618473 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.619035 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.619039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.619041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.619130 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.619496 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.619500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.619501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.619693 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:17.619801 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.620325 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.620473 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 16:41:17.621469 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.608699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:37617 (local address 127.11.158.131:34395)
0926 16:41:17.609190 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.609234 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.610488 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.611065 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.611072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.611450 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:17.611949 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.611963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.612948 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.612952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.613901 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.613912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.614431 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.614443 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.614613 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.615245 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:17.615262 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:17.616762 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:17.618620 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:17.618627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:17.618643 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:17.618927 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:17.619222 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:17.619226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:17.619228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:17.619372 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:17.619798 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:17.619804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:17.620084 (+ 280us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.620278 (+ 194us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.620632 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260926 16:41:17.622617 31542 heartbeater.cc:347] Connected to a master server at 127.11.158.190:37617
I20260926 16:41:17.622841 31542 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:17.623298 31542 heartbeater.cc:511] Master 127.11.158.190:37617 requested a full tablet report, sending...
I20260926 16:41:17.624457 31083 ts_manager.cc:194] Registered new tserver with Master: ea49997c1789466c8af351df0692087a (127.11.158.131:35653)
I20260926 16:41:17.625118 31083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.158.131@KRBTEST.COM'} at 127.11.158.131:34395
I20260926 16:41:17.626868 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:41:17.637035 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.629024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.11.158.190:37617)
0926 16:41:17.629167 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.629172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.629473 (+ 301us) server_negotiation.cc:407] Connection header received
0926 16:41:17.629602 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.629604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.629647 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.629717 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:17.630417 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.630922 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.632045 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.632265 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.632418 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.633888 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:17.633891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:17.633893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:17.633918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:17.635290 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.635736 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.635739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.635740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.635820 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:17.636083 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:17.636085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:17.636086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:17.636307 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:17.636487 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.636731 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.636837 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:41:17.639935 31083 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:41:17.643152 31083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:41:17.658970 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.653207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.11.158.129:40557)
0926 16:41:17.653611 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.653626 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.653646 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:41:17.653713 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.653718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.653896 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.654296 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:17.654440 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.655496 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.657439 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.658301 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.658455 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.658588 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.658771 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.658851 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":84,"threads_started":1}
I20260926 16:41:17.658968 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.652975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:40557 (local address 127.0.0.1:53362)
0926 16:41:17.653235 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.653249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.653358 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.654051 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.654054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.654072 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:17.654161 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.654167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.655658 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.655661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.657312 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.657324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.658445 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.658449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.658566 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.658611 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.658735 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.658842 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":208,"thread_start_us":83,"threads_started":1}
I20260926 16:41:17.659663 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.652208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:37915 (local address 127.0.0.1:49180)
0926 16:41:17.652582 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.652639 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.652773 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.653670 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.653677 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.653700 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:17.653944 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.653960 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.655695 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.655698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.657194 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.657203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.659211 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.659215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.659318 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.659353 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.659440 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.659520 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":149,"thread_start_us":99,"threads_started":1}
I20260926 16:41:17.659929 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.652438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.11.158.130:37915)
0926 16:41:17.653143 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.653155 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.653179 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:41:17.653258 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.653264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.653485 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.654109 (+ 624us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:17.654267 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.655494 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.657319 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.658301 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.658455 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.658588 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.659477 (+ 889us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.659593 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":170,"threads_started":1}
I20260926 16:41:17.660247 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.652720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:35653 (local address 127.0.0.1:33048)
0926 16:41:17.653821 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.653838 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.653943 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.654439 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.654442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.654462 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:17.654570 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.654580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.655616 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.655620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.657461 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.657473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.659835 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.659839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.659931 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.659957 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.660040 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.660113 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":421,"thread_start_us":91,"threads_started":1}
I20260926 16:41:17.660714 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.653455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.11.158.131:35653)
0926 16:41:17.654043 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.654058 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.654092 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:41:17.654157 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.654160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.654298 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.654460 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:17.654708 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.655466 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.657575 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.658302 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.658456 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.658591 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.660452 (+ 1861us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.660567 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":134,"threads_started":1}
I20260926 16:41:17.662508 31342 tablet_service.cc:1467] Processing CreateTablet for tablet 14a6da75dbdf4e1f97264d5f079e6420 (DEFAULT_TABLE table=test-table [id=2670da278223410bb4cef09af809965f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:17.662542 31208 tablet_service.cc:1467] Processing CreateTablet for tablet 14a6da75dbdf4e1f97264d5f079e6420 (DEFAULT_TABLE table=test-table [id=2670da278223410bb4cef09af809965f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:17.663582 31208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a6da75dbdf4e1f97264d5f079e6420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:17.663581 31342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a6da75dbdf4e1f97264d5f079e6420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:17.663450 31476 tablet_service.cc:1467] Processing CreateTablet for tablet 14a6da75dbdf4e1f97264d5f079e6420 (DEFAULT_TABLE table=test-table [id=2670da278223410bb4cef09af809965f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:17.664408 31476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a6da75dbdf4e1f97264d5f079e6420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:17.669669 31559 tablet_bootstrap.cc:492] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Bootstrap starting.
I20260926 16:41:17.670885 31560 tablet_bootstrap.cc:492] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf: Bootstrap starting.
I20260926 16:41:17.671854 31561 tablet_bootstrap.cc:492] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Bootstrap starting.
I20260926 16:41:17.672263 31559 tablet_bootstrap.cc:654] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:17.673139 31560 tablet_bootstrap.cc:654] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:17.673671 31561 tablet_bootstrap.cc:654] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:17.673643 31559 log.cc:824] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:17.674335 31560 log.cc:824] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:17.674969 31561 log.cc:824] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:17.676242 31559 tablet_bootstrap.cc:492] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: No bootstrap required, opened a new log
I20260926 16:41:17.676242 31560 tablet_bootstrap.cc:492] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf: No bootstrap required, opened a new log
I20260926 16:41:17.676468 31560 ts_tablet_manager.cc:1397] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:41:17.676468 31559 ts_tablet_manager.cc:1397] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:41:17.676848 31561 tablet_bootstrap.cc:492] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: No bootstrap required, opened a new log
I20260926 16:41:17.677052 31561 ts_tablet_manager.cc:1397] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:41:17.679994 31561 raft_consensus.cc:348] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.680267 31561 raft_consensus.cc:374] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:17.680452 31561 raft_consensus.cc:729] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea49997c1789466c8af351df0692087a, State: Initialized, Role: FOLLOWER
I20260926 16:41:17.680438 31560 raft_consensus.cc:348] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.680706 31560 raft_consensus.cc:374] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:17.680579 31559 raft_consensus.cc:348] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.680753 31560 raft_consensus.cc:729] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f846f400fdd54285a37bb5ea5e0caadf, State: Initialized, Role: FOLLOWER
I20260926 16:41:17.680785 31559 raft_consensus.cc:374] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:17.680830 31559 raft_consensus.cc:729] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 720abcffdcee4c86bd9bbaaf7da8c9ba, State: Initialized, Role: FOLLOWER
I20260926 16:41:17.681021 31561 consensus_queue.cc:260] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.681249 31559 consensus_queue.cc:260] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.681852 31542 heartbeater.cc:503] Master 127.11.158.190:37617 was elected leader, sending a full tablet report...
I20260926 16:41:17.681249 31560 consensus_queue.cc:260] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.682133 31561 ts_tablet_manager.cc:1428] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260926 16:41:17.682309 31559 ts_tablet_manager.cc:1428] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:41:17.682636 31560 ts_tablet_manager.cc:1428] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:41:17.682688 31408 heartbeater.cc:503] Master 127.11.158.190:37617 was elected leader, sending a full tablet report...
I20260926 16:41:17.682963 31274 heartbeater.cc:503] Master 127.11.158.190:37617 was elected leader, sending a full tablet report...
I20260926 16:41:17.790855 31566 raft_consensus.cc:482] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:41:17.791137 31566 raft_consensus.cc:504] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.792431 31566 leader_election.cc:288] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f846f400fdd54285a37bb5ea5e0caadf (127.11.158.129:40557), ea49997c1789466c8af351df0692087a (127.11.158.131:35653)
I20260926 16:41:17.796685 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.792749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:57717 (local address 127.11.158.129:40557)
0926 16:41:17.792888 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.792893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.792909 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:41:17.793008 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.793011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.793064 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.793160 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:17.793611 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.794244 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.795470 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.796076 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.796172 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.796215 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.796485 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.796548 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:41:17.796710 31410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.792646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:40557 (local address 127.11.158.130:57717)
0926 16:41:17.792760 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.792777 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.792897 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.793230 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.793234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.793264 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:17.793476 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.793487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.794349 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.794353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.795320 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.795331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.796205 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.796209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.796318 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.796371 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.796463 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.796545 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260926 16:41:17.797128 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.792723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:35653 (local address 127.11.158.130:39819)
0926 16:41:17.793141 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.793155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:17.793225 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:17.793532 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:17.793534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:17.793571 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:17.793699 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.793705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.794833 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.794836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.795801 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:17.795812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.796712 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.796719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:17.796822 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.796852 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 16:41:17.796926 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 16:41:17.796999 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":172,"threads_started":1}
I20260926 16:41:17.797186 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.792998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:39819 (local address 127.11.158.131:35653)
0926 16:41:17.793249 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.793254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.793292 (+ 38us) server_negotiation.cc:407] Connection header received
0926 16:41:17.793354 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.793357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.793413 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.793518 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:17.793837 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.794553 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.795952 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.796586 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.796698 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.796757 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.796991 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.797070 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:41:17.797489 31228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a6da75dbdf4e1f97264d5f079e6420" candidate_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f846f400fdd54285a37bb5ea5e0caadf" is_pre_election: true
I20260926 16:41:17.797816 31228 raft_consensus.cc:2454] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 720abcffdcee4c86bd9bbaaf7da8c9ba in term 0.
I20260926 16:41:17.797843 31496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a6da75dbdf4e1f97264d5f079e6420" candidate_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea49997c1789466c8af351df0692087a" is_pre_election: true
I20260926 16:41:17.798123 31496 raft_consensus.cc:2454] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 720abcffdcee4c86bd9bbaaf7da8c9ba in term 0.
I20260926 16:41:17.798382 31295 leader_election.cc:302] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 720abcffdcee4c86bd9bbaaf7da8c9ba, f846f400fdd54285a37bb5ea5e0caadf; no voters:
I20260926 16:41:17.798713 31566 raft_consensus.cc:2781] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:41:17.798795 31566 raft_consensus.cc:482] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:41:17.798827 31566 raft_consensus.cc:3037] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:17.799934 31566 raft_consensus.cc:504] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.800319 31566 leader_election.cc:288] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [CANDIDATE]: Term 1 election: Requested vote from peers f846f400fdd54285a37bb5ea5e0caadf (127.11.158.129:40557), ea49997c1789466c8af351df0692087a (127.11.158.131:35653)
I20260926 16:41:17.800916 31228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a6da75dbdf4e1f97264d5f079e6420" candidate_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f846f400fdd54285a37bb5ea5e0caadf"
I20260926 16:41:17.801090 31228 raft_consensus.cc:3037] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:17.801054 31496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a6da75dbdf4e1f97264d5f079e6420" candidate_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea49997c1789466c8af351df0692087a"
I20260926 16:41:17.801241 31496 raft_consensus.cc:3037] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:17.802327 31228 raft_consensus.cc:2454] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 720abcffdcee4c86bd9bbaaf7da8c9ba in term 1.
I20260926 16:41:17.802358 31496 raft_consensus.cc:2454] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 720abcffdcee4c86bd9bbaaf7da8c9ba in term 1.
I20260926 16:41:17.802719 31295 leader_election.cc:302] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 720abcffdcee4c86bd9bbaaf7da8c9ba, f846f400fdd54285a37bb5ea5e0caadf; no voters:
I20260926 16:41:17.802928 31566 raft_consensus.cc:2781] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:17.803186 31566 raft_consensus.cc:686] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 1 LEADER]: Becoming Leader. State: Replica: 720abcffdcee4c86bd9bbaaf7da8c9ba, State: Running, Role: LEADER
I20260926 16:41:17.803550 31566 consensus_queue.cc:237] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } }
I20260926 16:41:17.807129 31082 catalog_manager.cc:5495] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 720abcffdcee4c86bd9bbaaf7da8c9ba (127.11.158.130). New cstate: current_term: 1 leader_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "720abcffdcee4c86bd9bbaaf7da8c9ba" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 37915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 } health_report { overall_health: UNKNOWN } } }
I20260926 16:41:17.849597 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.845880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49182 (local address 127.11.158.130:37915)
0926 16:41:17.846038 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:17.846041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:17.846054 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:41:17.846129 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:17.846132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:17.846187 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:17.846306 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:17.846562 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.847077 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.848019 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:17.848243 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:17.848355 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:17.848740 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:17.849208 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:17.849326 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:17.849375 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 16:41:17.849455 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:41:17.855722 31228 raft_consensus.cc:1261] T 14a6da75dbdf4e1f97264d5f079e6420 P f846f400fdd54285a37bb5ea5e0caadf [term 1 FOLLOWER]: Refusing update from remote peer 720abcffdcee4c86bd9bbaaf7da8c9ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:17.855722 31496 raft_consensus.cc:1261] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 1 FOLLOWER]: Refusing update from remote peer 720abcffdcee4c86bd9bbaaf7da8c9ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:17.856492 31566 consensus_queue.cc:1035] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea49997c1789466c8af351df0692087a" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 35653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:17.856714 31569 consensus_queue.cc:1035] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "f846f400fdd54285a37bb5ea5e0caadf" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:41:17.857333 31543 tablet.cc:2369] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:41:17.864142 31572 mvcc.cc:205] Tried to move back new op lower bound from 7333645835689852928 to 7333645835485941760. Current Snapshot: MvccSnapshot[applied={T|T < 7333645835689852928}]
I20260926 16:41:17.865540 31574 mvcc.cc:205] Tried to move back new op lower bound from 7333645835689852928 to 7333645835485941760. Current Snapshot: MvccSnapshot[applied={T|T < 7333645835689852928}]
I20260926 16:41:17.873600 31573 mvcc.cc:205] Tried to move back new op lower bound from 7333645835689852928 to 7333645835485941760. Current Snapshot: MvccSnapshot[applied={T|T < 7333645835689852928 or (T in {7333645835689852928})}]
I20260926 16:41:17.874987 31083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:17.875160 31083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:17.877820 31083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7be3fcfdb7a4d9d94b006f73736b766: Sending DeleteTablet for 3 replicas of tablet 14a6da75dbdf4e1f97264d5f079e6420
I20260926 16:41:17.878780 31342 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a6da75dbdf4e1f97264d5f079e6420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:17 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:49180
I20260926 16:41:17.879323 31581 tablet_replica.cc:330] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: stopping tablet replica
I20260926 16:41:17.879311 31476 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a6da75dbdf4e1f97264d5f079e6420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:17 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:33048
I20260926 16:41:17.879810 31581 raft_consensus.cc:2229] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:41:17.879966 31208 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a6da75dbdf4e1f97264d5f079e6420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:17 UTC) from {username='kudu', principal='kudu/127.11.158.190@KRBTEST.COM'} at 127.0.0.1:53362
I20260926 16:41:17.880107 31582 tablet_replica.cc:330] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: stopping tablet replica
I20260926 16:41:17.880425 31582 raft_consensus.cc:2229] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:17.880807 31581 raft_consensus.cc:2258] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:17.880883 31582 raft_consensus.cc:2258] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:17.880721 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31145
I20260926 16:41:17.883474 31581 ts_tablet_manager.cc:1905] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:17.883759 31582 ts_tablet_manager.cc:1905] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:17.886786 31582 ts_tablet_manager.cc:1918] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:17.886979 31582 log.cc:1196] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/14a6da75dbdf4e1f97264d5f079e6420
I20260926 16:41:17.887368 31582 ts_tablet_manager.cc:1939] T 14a6da75dbdf4e1f97264d5f079e6420 P ea49997c1789466c8af351df0692087a: Deleting consensus metadata
I20260926 16:41:17.888053 31581 ts_tablet_manager.cc:1918] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:17.888167 31581 log.cc:1196] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/14a6da75dbdf4e1f97264d5f079e6420
I20260926 16:41:17.888543 31581 ts_tablet_manager.cc:1939] T 14a6da75dbdf4e1f97264d5f079e6420 P 720abcffdcee4c86bd9bbaaf7da8c9ba: Deleting consensus metadata
I20260926 16:41:17.889034 31070 catalog_manager.cc:4841] TS ea49997c1789466c8af351df0692087a (127.11.158.131:35653): tablet 14a6da75dbdf4e1f97264d5f079e6420 (table test-table [id=2670da278223410bb4cef09af809965f]) successfully deleted
I20260926 16:41:17.889758 31067 catalog_manager.cc:4841] TS 720abcffdcee4c86bd9bbaaf7da8c9ba (127.11.158.130:37915): tablet 14a6da75dbdf4e1f97264d5f079e6420 (table test-table [id=2670da278223410bb4cef09af809965f]) successfully deleted
W20260926 16:41:17.890228 31067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:40557 (error 108)
I20260926 16:41:17.890447 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31279
I20260926 16:41:17.893790 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:17.893427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:40557 (local address 127.0.0.1:53370)
0926 16:41:17.893593 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:17.893689 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260926 16:41:17.894273 31067 catalog_manager.cc:4584] TS f846f400fdd54285a37bb5ea5e0caadf (127.11.158.129:40557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14a6da75dbdf4e1f97264d5f079e6420: Network error: Client connection negotiation failed: client connection to 127.11.158.129:40557: connect: Connection refused (error 111)
I20260926 16:41:17.898046 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31413
I20260926 16:41:17.904057 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31051
2026-09-26T16:41:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2383 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T16:41:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:17Z Disabled control of system clock
I20260926 16:41:17.935786 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38001
--webserver_interface=127.11.158.190
--webserver_port=0
--builtin_ntp_servers=127.11.158.148:35733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:41:18.049449 31588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:18.049744 31588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:18.049782 31588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:18.054234 31588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:18.054318 31588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:18.054335 31588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:18.054354 31588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:18.054370 31588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:18.059589 31588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:18.060961 31588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:18.062429 31588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:18.071590 31588 server_base.cc:1034] running on GCE node
W20260926 16:41:18.071646 31594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:18.071640 31593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:18.071590 31596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:18.072575 31588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:18.073733 31588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:18.074960 31588 hybrid_clock.cc:648] HybridClock initialized: now 1790440878074947 us; error 28 us; skew 500 ppm
I20260926 16:41:18.076983 31588 webserver.cc:460] Webserver started at http://127.11.158.190:34003/ using document root <none> and password file <none>
I20260926 16:41:18.077590 31588 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:18.077687 31588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:18.077936 31588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:18.080581 31588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "3e3de03be00946a497fa5ad5b885af0a"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "42241e37fdf6903656d89f538e8ecfaa"
server_key_iv: "7b1b6a2b1c8fbdc7bf17e85c43251194"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.081336 31588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "3e3de03be00946a497fa5ad5b885af0a"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "42241e37fdf6903656d89f538e8ecfaa"
server_key_iv: "7b1b6a2b1c8fbdc7bf17e85c43251194"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.086015 31588 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:41:18.089424 31602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.090711 31588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:41:18.090853 31588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "3e3de03be00946a497fa5ad5b885af0a"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "42241e37fdf6903656d89f538e8ecfaa"
server_key_iv: "7b1b6a2b1c8fbdc7bf17e85c43251194"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.090981 31588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:18.101744 31588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:18.102440 31588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:18.102641 31588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:18.109576 31588 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:38001
I20260926 16:41:18.109586 31654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:38001 every 8 connection(s)
I20260926 16:41:18.110594 31588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:18.111936 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31588
I20260926 16:41:18.112114 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:18.112474 11898 external_mini_cluster.cc:1428] Setting key 680e341dd7dcba1c7cf2b579a4a4e580
I20260926 16:41:18.114379 31655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:18.121443 31655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a: Bootstrap starting.
I20260926 16:41:18.123289 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.113880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57386 (local address 127.11.158.190:38001)
0926 16:41:18.114584 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.114604 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.114648 (+ 44us) server_negotiation.cc:407] Connection header received
0926 16:41:18.115510 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.115532 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.116037 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.116433 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:18.116892 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.118410 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.119358 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.120048 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.120223 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.121153 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:18.121159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:18.121175 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:18.121694 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:18.121766 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:18.121792 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:18.121901 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.122040 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.122323 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":127,"threads_started":1}
I20260926 16:41:18.124320 31655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:18.125124 31655 log.cc:824] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:18.127310 31655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a: No bootstrap required, opened a new log
I20260926 16:41:18.131029 31655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3de03be00946a497fa5ad5b885af0a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38001 } }
I20260926 16:41:18.131289 31655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:18.131325 31655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3de03be00946a497fa5ad5b885af0a, State: Initialized, Role: FOLLOWER
I20260926 16:41:18.131815 31655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3de03be00946a497fa5ad5b885af0a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38001 } }
I20260926 16:41:18.131950 31655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:18.132051 31655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:18.132187 31655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:18.133427 31655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3de03be00946a497fa5ad5b885af0a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38001 } }
I20260926 16:41:18.133802 31655 leader_election.cc:302] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e3de03be00946a497fa5ad5b885af0a; no voters:
I20260926 16:41:18.134147 31655 leader_election.cc:288] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:18.134282 31660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:18.134578 31660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [term 1 LEADER]: Becoming Leader. State: Replica: 3e3de03be00946a497fa5ad5b885af0a, State: Running, Role: LEADER
I20260926 16:41:18.134902 31660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3de03be00946a497fa5ad5b885af0a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38001 } }
I20260926 16:41:18.135497 31655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:18.136612 31662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e3de03be00946a497fa5ad5b885af0a. Latest consensus state: current_term: 1 leader_uuid: "3e3de03be00946a497fa5ad5b885af0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3de03be00946a497fa5ad5b885af0a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38001 } } }
I20260926 16:41:18.136744 31662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:18.137290 31661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e3de03be00946a497fa5ad5b885af0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3de03be00946a497fa5ad5b885af0a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38001 } } }
I20260926 16:41:18.137401 31661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:18.137271 31669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:18.141103 31669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:18.147358 31669 catalog_manager.cc:1347] Generated new cluster ID: 6fd8e919414f442da3e5584789b8e412
I20260926 16:41:18.147464 31669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:18.181986 31669 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:18.183363 31669 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:18.190738 31669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a: Generated new TSK 0
I20260926 16:41:18.191748 31669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:18.198767 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.129:0
--local_ip_for_outbound_sockets=127.11.158.129
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38001
--builtin_ntp_servers=127.11.158.148:35733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:41:18.322553 31679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:18.322860 31679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:18.322901 31679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:18.327126 31679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:18.327191 31679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:18.327250 31679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.129
I20260926 16:41:18.332274 31679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.158.129
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.129
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:18.333423 31679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:18.334798 31679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:18.345731 31685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:18.345752 31679 server_base.cc:1034] running on GCE node
W20260926 16:41:18.345649 31684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:18.346061 31687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:18.346567 31679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:18.347287 31679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:18.348453 31679 hybrid_clock.cc:648] HybridClock initialized: now 1790440878348438 us; error 28 us; skew 500 ppm
I20260926 16:41:18.350456 31679 webserver.cc:460] Webserver started at http://127.11.158.129:35831/ using document root <none> and password file <none>
I20260926 16:41:18.351082 31679 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:18.351181 31679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:18.351430 31679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:18.353670 31679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data/instance:
uuid: "28814c1c0b134f8a94c7c7fb81607f5a"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "0e2612af8abb6c0df94159c9d1e87018"
server_key_iv: "58c7b7131c121a30310482278ec2ab7f"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.354254 31679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance:
uuid: "28814c1c0b134f8a94c7c7fb81607f5a"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "0e2612af8abb6c0df94159c9d1e87018"
server_key_iv: "58c7b7131c121a30310482278ec2ab7f"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.358003 31679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:41:18.360622 31693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.361930 31679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:18.362071 31679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/wal
uuid: "28814c1c0b134f8a94c7c7fb81607f5a"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "0e2612af8abb6c0df94159c9d1e87018"
server_key_iv: "58c7b7131c121a30310482278ec2ab7f"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.362195 31679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:18.381994 31679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:18.382761 31679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:18.382974 31679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:18.383613 31679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:18.384686 31679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:18.384757 31679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.384835 31679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:18.384877 31679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.394528 31679 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.129:41483
I20260926 16:41:18.394644 31806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.129:41483 every 8 connection(s)
I20260926 16:41:18.395689 31679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/data/info.pb
I20260926 16:41:18.396270 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31679
I20260926 16:41:18.396423 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-0/wal/instance
I20260926 16:41:18.396744 11898 external_mini_cluster.cc:1428] Setting key 240c3885a0914627d36b73e3fbc25a32
I20260926 16:41:18.398908 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.130:0
--local_ip_for_outbound_sockets=127.11.158.130
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38001
--builtin_ntp_servers=127.11.158.148:35733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:41:18.407183 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.397730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.129:59421 (local address 127.11.158.190:38001)
0926 16:41:18.397888 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.397893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.399103 (+ 1210us) server_negotiation.cc:407] Connection header received
0926 16:41:18.400830 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.400833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.400883 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.400956 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:18.402225 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.402833 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.404929 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.405154 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.405246 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.406640 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:18.406644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:18.406646 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:18.406660 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:18.406678 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:18.406686 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:18.406747 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.406985 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.407058 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:41:18.408509 31809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.398217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38001 (local address 127.11.158.129:59421)
0926 16:41:18.398945 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.399004 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.400578 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.401185 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.401198 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.401320 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:18.402054 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.402076 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.403001 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.403008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.404774 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.404789 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.405329 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.405337 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.405578 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.406303 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:18.406306 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:18.406368 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:18.406467 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:18.406779 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:18.406790 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:18.406806 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.406960 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.407306 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":149,"threads_started":1}
I20260926 16:41:18.409935 31807 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38001
I20260926 16:41:18.410197 31807 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:18.410754 31807 heartbeater.cc:511] Master 127.11.158.190:38001 requested a full tablet report, sending...
I20260926 16:41:18.412294 31618 ts_manager.cc:194] Registered new tserver with Master: 28814c1c0b134f8a94c7c7fb81607f5a (127.11.158.129:41483)
I20260926 16:41:18.413560 31618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.129:59421
W20260926 16:41:18.515697 31810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:18.516010 31810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:18.516084 31810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:18.520220 31810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:18.520341 31810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:18.520474 31810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.130
I20260926 16:41:18.525528 31810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.158.130
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.130
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:18.526724 31810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:18.528151 31810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:18.538506 31816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:18.538538 31818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:18.538529 31815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:18.538576 31810 server_base.cc:1034] running on GCE node
I20260926 16:41:18.539268 31810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:18.540024 31810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:18.541240 31810 hybrid_clock.cc:648] HybridClock initialized: now 1790440878541206 us; error 44 us; skew 500 ppm
I20260926 16:41:18.543205 31810 webserver.cc:460] Webserver started at http://127.11.158.130:36153/ using document root <none> and password file <none>
I20260926 16:41:18.543819 31810 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:18.543887 31810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:18.544113 31810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:18.546290 31810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data/instance:
uuid: "bce3d5351aa448979fe1224bcbde3dbe"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "893fcdba19f79d711de71fa7ad346c92"
server_key_iv: "60431ef0ea0b36a05cb115a38eab8e8b"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.546892 31810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance:
uuid: "bce3d5351aa448979fe1224bcbde3dbe"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "893fcdba19f79d711de71fa7ad346c92"
server_key_iv: "60431ef0ea0b36a05cb115a38eab8e8b"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.550812 31810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:41:18.553534 31824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.554888 31810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:18.555027 31810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal
uuid: "bce3d5351aa448979fe1224bcbde3dbe"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "893fcdba19f79d711de71fa7ad346c92"
server_key_iv: "60431ef0ea0b36a05cb115a38eab8e8b"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.555150 31810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:18.591018 31810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:18.591774 31810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:18.591987 31810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:18.592679 31810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:18.593648 31810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:18.593696 31810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.593735 31810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:18.593817 31810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.602507 31810 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.130:44387
I20260926 16:41:18.602587 31937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.130:44387 every 8 connection(s)
I20260926 16:41:18.603449 31810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/data/info.pb
I20260926 16:41:18.606598 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31810
I20260926 16:41:18.606726 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal/instance
I20260926 16:41:18.607012 11898 external_mini_cluster.cc:1428] Setting key a315e79033ddb75b37cd358d871e46b8
I20260926 16:41:18.608973 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.158.131:0
--local_ip_for_outbound_sockets=127.11.158.131
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38001
--builtin_ntp_servers=127.11.158.148:35733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:41:18.613174 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.605432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:54655 (local address 127.11.158.190:38001)
0926 16:41:18.605571 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.605574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.606549 (+ 975us) server_negotiation.cc:407] Connection header received
0926 16:41:18.608193 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.608197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.608254 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.608387 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:18.609383 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.609906 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.611082 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.611314 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.611428 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.612643 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:18.612647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:18.612650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:18.612671 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:18.612692 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:18.612700 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:18.612777 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.612959 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.613027 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:41:18.614100 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.605737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38001 (local address 127.11.158.130:54655)
0926 16:41:18.606399 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.606440 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.607982 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.608575 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.608582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.608669 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:18.609234 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.609250 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.610039 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.610042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.610947 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.610957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.611475 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.611479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.611647 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.612308 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:18.612310 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:18.612410 (+ 100us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:18.612480 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:18.612791 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:18.612803 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:18.612810 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.612958 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.613258 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260926 16:41:18.615404 31938 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38001
I20260926 16:41:18.615694 31938 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:18.616436 31938 heartbeater.cc:511] Master 127.11.158.190:38001 requested a full tablet report, sending...
I20260926 16:41:18.617729 31618 ts_manager.cc:194] Registered new tserver with Master: bce3d5351aa448979fe1224bcbde3dbe (127.11.158.130:44387)
I20260926 16:41:18.618647 31618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.130:54655
W20260926 16:41:18.722635 31941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:18.722949 31941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:18.722997 31941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:18.727211 31941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:18.727267 31941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:18.727324 31941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.158.131
I20260926 16:41:18.732074 31941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:35733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.158.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.158.131
--webserver_port=0
--tserver_master_addrs=127.11.158.190:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.158.131
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:18.733170 31941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:18.734537 31941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:18.744398 31949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:18.744413 31947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:18.744370 31946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:18.744686 31941 server_base.cc:1034] running on GCE node
I20260926 16:41:18.745249 31941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:18.745823 31941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:18.746987 31941 hybrid_clock.cc:648] HybridClock initialized: now 1790440878746964 us; error 28 us; skew 500 ppm
I20260926 16:41:18.748804 31941 webserver.cc:460] Webserver started at http://127.11.158.131:46531/ using document root <none> and password file <none>
I20260926 16:41:18.749336 31941 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:18.749393 31941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:18.749552 31941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:18.751621 31941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data/instance:
uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "32803f0fa896d8629a2cd6efb788addb"
server_key_iv: "6a6498d84bc79d7491453f907dc40c7e"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.752168 31941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance:
uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "32803f0fa896d8629a2cd6efb788addb"
server_key_iv: "6a6498d84bc79d7491453f907dc40c7e"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.755867 31941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:18.758333 31955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.759691 31941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:41:18.759781 31941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal
uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f"
format_stamp: "Formatted at 2026-09-26 16:41:18 on dist-test-slave-p39x"
server_key: "32803f0fa896d8629a2cd6efb788addb"
server_key_iv: "6a6498d84bc79d7491453f907dc40c7e"
server_key_version: "encryptionkey@0"
I20260926 16:41:18.759910 31941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:18.786477 31941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:18.787209 31941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:18.787410 31941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:18.788024 31941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:41:18.789038 31941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:41:18.789085 31941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.789152 31941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:41:18.789196 31941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:18.798151 31941 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.131:34193
I20260926 16:41:18.798175 32068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.131:34193 every 8 connection(s)
I20260926 16:41:18.799083 31941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/data/info.pb
I20260926 16:41:18.805683 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 31941
I20260926 16:41:18.805787 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal/instance
I20260926 16:41:18.806047 11898 external_mini_cluster.cc:1428] Setting key 18aa152582bcf248b006fcc59da287f1
I20260926 16:41:18.808519 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.800907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.131:42409 (local address 127.11.158.190:38001)
0926 16:41:18.801080 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.801084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.802146 (+ 1062us) server_negotiation.cc:407] Connection header received
0926 16:41:18.803467 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.803469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.803521 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.803596 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:18.804526 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.805044 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.806288 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.806571 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.806673 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.807892 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:18.807895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:18.807897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:18.807908 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:18.807927 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:18.807934 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:18.807994 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.808203 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.808291 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 16:41:18.809299 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.801332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38001 (local address 127.11.158.131:42409)
0926 16:41:18.801971 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.802010 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.803277 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.803762 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.803769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.803853 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:18.804390 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.804402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.805151 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.805153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.806166 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.806176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.806759 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.806764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.807023 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.807609 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:41:18.807611 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:18.807650 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:41:18.807748 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:18.808017 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:18.808025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:18.808031 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.808159 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.808486 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":110,"threads_started":1}
I20260926 16:41:18.810382 32069 heartbeater.cc:347] Connected to a master server at 127.11.158.190:38001
I20260926 16:41:18.810591 32069 heartbeater.cc:464] Registering TS with master...
I20260926 16:41:18.811048 32069 heartbeater.cc:511] Master 127.11.158.190:38001 requested a full tablet report, sending...
I20260926 16:41:18.812017 31618 ts_manager.cc:194] Registered new tserver with Master: 3ce96e1ac5514b1e9ab09e79968e3f0f (127.11.158.131:34193)
I20260926 16:41:18.812635 31618 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.158.131:42409
I20260926 16:41:18.820542 11898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:41:18.826177 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.822551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.11.158.190:38001)
0926 16:41:18.822703 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.822710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.822843 (+ 133us) server_negotiation.cc:407] Connection header received
0926 16:41:18.822967 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.822969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.823012 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.823076 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:18.823282 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.823776 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.824728 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.824944 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.825021 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.825734 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:18.825737 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:18.825739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:18.825749 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:18.825766 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:18.825773 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:18.825830 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.825928 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.825985 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 16:41:18.828933 31618 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:41:18.831809 31618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:41:18.846509 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.841331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.11.158.131:34193)
0926 16:41:18.842147 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.842159 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.842180 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:41:18.842251 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.842256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.842429 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.842610 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:18.842826 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.843881 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.845303 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.845831 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.845977 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.846160 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.846247 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.846344 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":155,"threads_started":1}
I20260926 16:41:18.846503 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.841182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:34193 (local address 127.0.0.1:56276)
0926 16:41:18.841803 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.841816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.841905 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.842605 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.842611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.842633 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:18.842709 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.842715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.844005 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.844009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.845187 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.845194 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.845965 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.845969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.846068 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.846104 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.846217 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.846339 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":79,"threads_started":1}
I20260926 16:41:18.846784 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.840435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:41483 (local address 127.0.0.1:58802)
0926 16:41:18.840803 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.840854 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.840987 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.841884 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.841890 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.841914 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:18.842050 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.842057 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.843537 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.843542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.844895 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.844904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.846237 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.846241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.846443 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.846490 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.846558 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.846649 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":176,"thread_start_us":87,"threads_started":1}
I20260926 16:41:18.846896 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.841277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59356 (local address 127.11.158.130:44387)
0926 16:41:18.842014 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.842026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.842049 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:41:18.842120 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.842128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.842289 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.842435 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:18.842667 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.843467 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.845210 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.845979 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.846081 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.846210 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.846647 (+ 437us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.846736 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":115,"threads_started":1}
I20260926 16:41:18.847196 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.840866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.130:44387 (local address 127.0.0.1:59356)
0926 16:41:18.841658 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.841673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.841763 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.842411 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.842414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.842431 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:18.842536 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.842545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.843585 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.843588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.845074 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.845084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.846256 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.846259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.846432 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.846488 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.846558 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.847086 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":559,"thread_start_us":86,"threads_started":1}
I20260926 16:41:18.847404 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.840602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.11.158.129:41483)
0926 16:41:18.841205 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.841220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.841251 (+ 31us) server_negotiation.cc:407] Connection header received
0926 16:41:18.841333 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.841339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.841564 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.841733 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:18.842182 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.843418 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.845027 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.845870 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.845993 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.846169 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.846959 (+ 790us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.847071 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":134,"threads_started":1}
I20260926 16:41:18.850047 31741 tablet_service.cc:1467] Processing CreateTablet for tablet b3192fed81994cd3a5d3c7fd51090c54 (DEFAULT_TABLE table=test-table [id=12cbdfcd71c84a63904c3eda58a73e62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:18.850047 31872 tablet_service.cc:1467] Processing CreateTablet for tablet b3192fed81994cd3a5d3c7fd51090c54 (DEFAULT_TABLE table=test-table [id=12cbdfcd71c84a63904c3eda58a73e62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:18.851205 31872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3192fed81994cd3a5d3c7fd51090c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:18.851207 31741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3192fed81994cd3a5d3c7fd51090c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:18.850400 32003 tablet_service.cc:1467] Processing CreateTablet for tablet b3192fed81994cd3a5d3c7fd51090c54 (DEFAULT_TABLE table=test-table [id=12cbdfcd71c84a63904c3eda58a73e62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:41:18.854071 32003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3192fed81994cd3a5d3c7fd51090c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:18.858811 32086 tablet_bootstrap.cc:492] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: Bootstrap starting.
I20260926 16:41:18.860067 32087 tablet_bootstrap.cc:492] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Bootstrap starting.
I20260926 16:41:18.861356 32086 tablet_bootstrap.cc:654] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:18.861653 32088 tablet_bootstrap.cc:492] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Bootstrap starting.
I20260926 16:41:18.862200 32086 log.cc:824] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:18.862331 32087 tablet_bootstrap.cc:654] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:18.863126 32087 log.cc:824] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:18.863672 32088 tablet_bootstrap.cc:654] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:18.864521 32088 log.cc:824] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:18.864918 32086 tablet_bootstrap.cc:492] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: No bootstrap required, opened a new log
I20260926 16:41:18.865134 32086 ts_tablet_manager.cc:1397] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:41:18.865970 32087 tablet_bootstrap.cc:492] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: No bootstrap required, opened a new log
I20260926 16:41:18.866173 32087 ts_tablet_manager.cc:1397] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:41:18.868685 32086 raft_consensus.cc:348] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.868978 32086 raft_consensus.cc:374] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:18.869059 32086 raft_consensus.cc:729] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28814c1c0b134f8a94c7c7fb81607f5a, State: Initialized, Role: FOLLOWER
I20260926 16:41:18.869282 32087 raft_consensus.cc:348] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.869541 32087 raft_consensus.cc:374] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:18.869592 32087 raft_consensus.cc:729] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bce3d5351aa448979fe1224bcbde3dbe, State: Initialized, Role: FOLLOWER
I20260926 16:41:18.869735 32086 consensus_queue.cc:260] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.870047 32087 consensus_queue.cc:260] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.870703 31938 heartbeater.cc:503] Master 127.11.158.190:38001 was elected leader, sending a full tablet report...
I20260926 16:41:18.870853 32086 ts_tablet_manager.cc:1428] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:41:18.870949 32087 ts_tablet_manager.cc:1428] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:41:18.871193 31807 heartbeater.cc:503] Master 127.11.158.190:38001 was elected leader, sending a full tablet report...
I20260926 16:41:18.871635 32088 tablet_bootstrap.cc:492] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: No bootstrap required, opened a new log
I20260926 16:41:18.871857 32088 ts_tablet_manager.cc:1397] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260926 16:41:18.876051 32088 raft_consensus.cc:348] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.876302 32088 raft_consensus.cc:374] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:18.876401 32088 raft_consensus.cc:729] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce96e1ac5514b1e9ab09e79968e3f0f, State: Initialized, Role: FOLLOWER
I20260926 16:41:18.876885 32088 consensus_queue.cc:260] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.877580 32069 heartbeater.cc:503] Master 127.11.158.190:38001 was elected leader, sending a full tablet report...
I20260926 16:41:18.877907 32088 ts_tablet_manager.cc:1428] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
W20260926 16:41:18.897025 31808 tablet.cc:2369] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:41:18.933575 32093 raft_consensus.cc:482] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:41:18.933838 32093 raft_consensus.cc:504] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.935143 32093 leader_election.cc:288] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28814c1c0b134f8a94c7c7fb81607f5a (127.11.158.129:41483), 3ce96e1ac5514b1e9ab09e79968e3f0f (127.11.158.131:34193)
I20260926 16:41:18.939616 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.935606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:60767 (local address 127.11.158.131:34193)
0926 16:41:18.935742 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.935748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.935767 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:41:18.935834 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.935837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.935909 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.936023 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:18.936410 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.937115 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.938508 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.939016 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.939143 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.939193 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.939420 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.939492 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 16:41:18.939625 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.935591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.158.130:38473 (local address 127.11.158.129:41483)
0926 16:41:18.935720 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.935725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.936087 (+ 362us) server_negotiation.cc:407] Connection header received
0926 16:41:18.936198 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.936201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.936252 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.936408 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:41:18.936658 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.937268 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.938478 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.939024 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.939154 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.939201 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.939448 (+ 247us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.939513 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:41:18.939626 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.935457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.131:34193 (local address 127.11.158.130:60767)
0926 16:41:18.935597 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.935615 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.935718 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.936030 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.936032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.936070 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:18.936224 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.936231 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.937218 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.937221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.938366 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.938375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.939137 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.939140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.939259 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.939312 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.939373 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.939476 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":30}
I20260926 16:41:18.939640 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.935468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:41483 (local address 127.11.158.130:38473)
0926 16:41:18.935980 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:18.935995 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:18.936092 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:18.936415 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:18.936418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:18.936436 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:41:18.936546 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.936552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.937381 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.937383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.938343 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:18.938353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.939158 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.939161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:18.939305 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.939322 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 16:41:18.939417 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 16:41:18.939485 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":156,"thread_start_us":124,"threads_started":1}
I20260926 16:41:18.940480 32023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3192fed81994cd3a5d3c7fd51090c54" candidate_uuid: "bce3d5351aa448979fe1224bcbde3dbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" is_pre_election: true
I20260926 16:41:18.940480 31761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3192fed81994cd3a5d3c7fd51090c54" candidate_uuid: "bce3d5351aa448979fe1224bcbde3dbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" is_pre_election: true
I20260926 16:41:18.940795 31761 raft_consensus.cc:2454] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bce3d5351aa448979fe1224bcbde3dbe in term 0.
I20260926 16:41:18.940795 32023 raft_consensus.cc:2454] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bce3d5351aa448979fe1224bcbde3dbe in term 0.
I20260926 16:41:18.941289 31828 leader_election.cc:302] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 28814c1c0b134f8a94c7c7fb81607f5a, 3ce96e1ac5514b1e9ab09e79968e3f0f, bce3d5351aa448979fe1224bcbde3dbe; no voters:
I20260926 16:41:18.941542 32093 raft_consensus.cc:2781] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:41:18.941633 32093 raft_consensus.cc:482] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:41:18.941679 32093 raft_consensus.cc:3037] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:18.942703 32093 raft_consensus.cc:504] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.943058 32093 leader_election.cc:288] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [CANDIDATE]: Term 1 election: Requested vote from peers 28814c1c0b134f8a94c7c7fb81607f5a (127.11.158.129:41483), 3ce96e1ac5514b1e9ab09e79968e3f0f (127.11.158.131:34193)
I20260926 16:41:18.943363 31761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3192fed81994cd3a5d3c7fd51090c54" candidate_uuid: "bce3d5351aa448979fe1224bcbde3dbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28814c1c0b134f8a94c7c7fb81607f5a"
I20260926 16:41:18.943490 32023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3192fed81994cd3a5d3c7fd51090c54" candidate_uuid: "bce3d5351aa448979fe1224bcbde3dbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f"
I20260926 16:41:18.943512 31761 raft_consensus.cc:3037] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:18.943643 32023 raft_consensus.cc:3037] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:18.944770 32023 raft_consensus.cc:2454] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bce3d5351aa448979fe1224bcbde3dbe in term 1.
I20260926 16:41:18.944785 31761 raft_consensus.cc:2454] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bce3d5351aa448979fe1224bcbde3dbe in term 1.
I20260926 16:41:18.945091 31827 leader_election.cc:302] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce96e1ac5514b1e9ab09e79968e3f0f, bce3d5351aa448979fe1224bcbde3dbe; no voters:
I20260926 16:41:18.945273 32093 raft_consensus.cc:2781] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:18.945498 32093 raft_consensus.cc:686] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 1 LEADER]: Becoming Leader. State: Replica: bce3d5351aa448979fe1224bcbde3dbe, State: Running, Role: LEADER
I20260926 16:41:18.945828 32093 consensus_queue.cc:237] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } }
I20260926 16:41:18.948995 31618 catalog_manager.cc:5495] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe reported cstate change: term changed from 0 to 1, leader changed from <none> to bce3d5351aa448979fe1224bcbde3dbe (127.11.158.130). New cstate: current_term: 1 leader_uuid: "bce3d5351aa448979fe1224bcbde3dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bce3d5351aa448979fe1224bcbde3dbe" member_type: VOTER last_known_addr { host: "127.11.158.130" port: 44387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 } health_report { overall_health: UNKNOWN } } }
I20260926 16:41:18.971843 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:18.968249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.11.158.130:44387)
0926 16:41:18.968420 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:18.968427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:18.968440 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:41:18.968540 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:18.968543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:18.968601 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:18.968718 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:18.968980 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.969484 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.970341 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:18.970565 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:18.970679 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:18.970997 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:18.971418 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:18.971554 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:18.971616 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 16:41:18.971673 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:41:18.978426 31761 raft_consensus.cc:1261] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 1 FOLLOWER]: Refusing update from remote peer bce3d5351aa448979fe1224bcbde3dbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:18.978426 32023 raft_consensus.cc:1261] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 1 FOLLOWER]: Refusing update from remote peer bce3d5351aa448979fe1224bcbde3dbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:18.979377 32096 consensus_queue.cc:1035] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "28814c1c0b134f8a94c7c7fb81607f5a" member_type: VOTER last_known_addr { host: "127.11.158.129" port: 41483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:18.979633 32093 consensus_queue.cc:1035] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce96e1ac5514b1e9ab09e79968e3f0f" member_type: VOTER last_known_addr { host: "127.11.158.131" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:18.990914 32100 mvcc.cc:205] Tried to move back new op lower bound from 7333645840286257152 to 7333645840164876288. Current Snapshot: MvccSnapshot[applied={T|T < 7333645840286257152 or (T in {7333645840286257152})}]
I20260926 16:41:18.998929 31619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:18.999114 31619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:41:19.001749 31619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e3de03be00946a497fa5ad5b885af0a: Sending DeleteTablet for 3 replicas of tablet b3192fed81994cd3a5d3c7fd51090c54
I20260926 16:41:19.002671 31872 tablet_service.cc:1514] Processing DeleteTablet for tablet b3192fed81994cd3a5d3c7fd51090c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:18 UTC) from {username='slave'} at 127.0.0.1:59356
I20260926 16:41:19.002724 31741 tablet_service.cc:1514] Processing DeleteTablet for tablet b3192fed81994cd3a5d3c7fd51090c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:18 UTC) from {username='slave'} at 127.0.0.1:58802
I20260926 16:41:19.003264 32109 tablet_replica.cc:330] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: stopping tablet replica
I20260926 16:41:19.003295 32110 tablet_replica.cc:330] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a: stopping tablet replica
I20260926 16:41:19.003269 32003 tablet_service.cc:1514] Processing DeleteTablet for tablet b3192fed81994cd3a5d3c7fd51090c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:41:18 UTC) from {username='slave'} at 127.0.0.1:56276
I20260926 16:41:19.003646 32110 raft_consensus.cc:2229] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:19.003916 32111 tablet_replica.cc:330] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: stopping tablet replica
I20260926 16:41:19.003998 32110 raft_consensus.cc:2258] T b3192fed81994cd3a5d3c7fd51090c54 P 28814c1c0b134f8a94c7c7fb81607f5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:19.004199 32111 raft_consensus.cc:2229] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:41:19.004526 32111 raft_consensus.cc:2258] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:19.004659 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31679
I20260926 16:41:19.004680 32109 raft_consensus.cc:2229] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:41:19.005666 32109 raft_consensus.cc:2258] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:41:19.006457 32111 ts_tablet_manager.cc:1905] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:19.007409 32109 ts_tablet_manager.cc:1905] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:41:19.009370 32111 ts_tablet_manager.cc:1918] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:19.009473 32111 log.cc:1196] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-2/wal/wals/b3192fed81994cd3a5d3c7fd51090c54
I20260926 16:41:19.009776 32111 ts_tablet_manager.cc:1939] T b3192fed81994cd3a5d3c7fd51090c54 P 3ce96e1ac5514b1e9ab09e79968e3f0f: Deleting consensus metadata
I20260926 16:41:19.011116 31605 catalog_manager.cc:4841] TS 3ce96e1ac5514b1e9ab09e79968e3f0f (127.11.158.131:34193): tablet b3192fed81994cd3a5d3c7fd51090c54 (table test-table [id=12cbdfcd71c84a63904c3eda58a73e62]) successfully deleted
I20260926 16:41:19.011236 32109 ts_tablet_manager.cc:1918] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:41:19.011336 32109 log.cc:1196] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Deleting WAL directory at /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440622937842-11898-0/minicluster-data/ts-1/wal/wals/b3192fed81994cd3a5d3c7fd51090c54
I20260926 16:41:19.011709 32109 ts_tablet_manager.cc:1939] T b3192fed81994cd3a5d3c7fd51090c54 P bce3d5351aa448979fe1224bcbde3dbe: Deleting consensus metadata
I20260926 16:41:19.013041 31605 catalog_manager.cc:4841] TS bce3d5351aa448979fe1224bcbde3dbe (127.11.158.130:44387): tablet b3192fed81994cd3a5d3c7fd51090c54 (table test-table [id=12cbdfcd71c84a63904c3eda58a73e62]) successfully deleted
W20260926 16:41:19.015635 31606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.158.129:41483 (error 108)
I20260926 16:41:19.015867 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31810
I20260926 16:41:19.017448 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:19.016978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.129:41483 (local address 127.0.0.1:58816)
0926 16:41:19.017217 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:19.017339 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.129:41483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260926 16:41:19.017835 31606 catalog_manager.cc:4584] TS 28814c1c0b134f8a94c7c7fb81607f5a (127.11.158.129:41483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3192fed81994cd3a5d3c7fd51090c54: Network error: Client connection negotiation failed: client connection to 127.11.158.129:41483: connect: Connection refused (error 111)
I20260926 16:41:19.022194 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31941
I20260926 16:41:19.027868 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 31588
2026-09-26T16:41:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1122 ms)
[----------] 6 tests from EncryptionPolicyTest (10431 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:19 dist-test-slave-p39x krb5kdc[32115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:19 dist-test-slave-p39x krb5kdc[32115](info): set up 2 sockets
Sep 26 16:41:19 dist-test-slave-p39x krb5kdc[32115](info): commencing operation
krb5kdc: starting...
W20260926 16:41:20.081123 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:20 dist-test-slave-p39x krb5kdc[32115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:20.208989 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41157
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:20.320322 32128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:20.320715 32128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:20.320802 32128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:20.325016 32128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:20.325127 32128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:20.325168 32128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:20.325229 32128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:20.325268 32128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:20.330832 32128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41157
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:20.332005 32128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:20.333093 32128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:20.342175 32128 server_base.cc:1034] running on GCE node
W20260926 16:41:20.342132 32133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:20.342132 32134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:20.342132 32136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:20.343008 32128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:20.344133 32128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:20.345306 32128 hybrid_clock.cc:648] HybridClock initialized: now 1790440880345301 us; error 72 us; skew 500 ppm
Sep 26 16:41:20 dist-test-slave-p39x krb5kdc[32115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440880, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:20.353869 32128 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:20.354897 32128 webserver.cc:460] Webserver started at http://127.0.0.1:40777/ using document root <none> and password file <none>
I20260926 16:41:20.355475 32128 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:20.355536 32128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:20.355710 32128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:20.357923 32128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2"
format_stamp: "Formatted at 2026-09-26 16:41:20 on dist-test-slave-p39x"
server_key: "1d56de9e5ef7b97ea8e59b1af946be69"
server_key_iv: "93f42dcadef0dda45cfb7bb4e1c5904b"
server_key_version: "encryptionkey@0"
I20260926 16:41:20.358453 32128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2"
format_stamp: "Formatted at 2026-09-26 16:41:20 on dist-test-slave-p39x"
server_key: "1d56de9e5ef7b97ea8e59b1af946be69"
server_key_iv: "93f42dcadef0dda45cfb7bb4e1c5904b"
server_key_version: "encryptionkey@0"
I20260926 16:41:20.362373 32128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:41:20.364910 32143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:20.366227 32128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:20.366370 32128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2"
format_stamp: "Formatted at 2026-09-26 16:41:20 on dist-test-slave-p39x"
server_key: "1d56de9e5ef7b97ea8e59b1af946be69"
server_key_iv: "93f42dcadef0dda45cfb7bb4e1c5904b"
server_key_version: "encryptionkey@0"
I20260926 16:41:20.366514 32128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:20.390349 32128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:20.393962 32128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:20.394212 32128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:20.402015 32128 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41157
I20260926 16:41:20.402042 32195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41157 every 8 connection(s)
I20260926 16:41:20.403069 32128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:20.405616 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 32128
I20260926 16:41:20.405790 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:20.405918 32196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:20.406184 11898 external_mini_cluster.cc:1428] Setting key 377cf4b474dd935482cfb130d36c9443
I20260926 16:41:20.412246 32196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2: Bootstrap starting.
I20260926 16:41:20.414633 32196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2: Neither blocks nor log segments found. Creating new log.
Sep 26 16:41:20 dist-test-slave-p39x krb5kdc[32115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:41:20.415592 32196 log.cc:824] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:20.417852 32196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2: No bootstrap required, opened a new log
I20260926 16:41:20.420918 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:20.407629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51199 (local address 127.0.0.1:41157)
0926 16:41:20.408096 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:20.408111 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:20.408147 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:41:20.409153 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:20.409171 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:20.409512 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:20.409856 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:20.410673 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:20.411663 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:20.412650 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:20.413124 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:20.413293 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:20.416134 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:20.416140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:20.416158 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:20.416191 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:20.417963 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:20.418521 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:20.418526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:20.418530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:20.418636 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:20.418982 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:20.418984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:20.418985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:20.419364 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:20.419526 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:20.419855 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 16:41:20.420098 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":153,"threads_started":1}
I20260926 16:41:20.422045 32196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41157 } }
I20260926 16:41:20.422333 32196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:20.422391 32196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ae0b07fd854b8d9e9a7acee7cf1fc2, State: Initialized, Role: FOLLOWER
I20260926 16:41:20.422843 32196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41157 } }
I20260926 16:41:20.423032 32196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:20.423099 32196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:20.423225 32196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:20.424100 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:20.424451 32196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41157 } }
I20260926 16:41:20.424784 32196 leader_election.cc:302] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57ae0b07fd854b8d9e9a7acee7cf1fc2; no voters:
I20260926 16:41:20.425130 32196 leader_election.cc:288] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:20.425203 32201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:20.425390 32201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [term 1 LEADER]: Becoming Leader. State: Replica: 57ae0b07fd854b8d9e9a7acee7cf1fc2, State: Running, Role: LEADER
I20260926 16:41:20.425678 32201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41157 } }
I20260926 16:41:20.426427 32196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:20.426822 32201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41157 } } }
I20260926 16:41:20.426949 32201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:20.427138 32201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57ae0b07fd854b8d9e9a7acee7cf1fc2. Latest consensus state: current_term: 1 leader_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ae0b07fd854b8d9e9a7acee7cf1fc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41157 } } }
I20260926 16:41:20.427234 32201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2 [sys.catalog]: This master's current role is: LEADER
W20260926 16:41:20.431336 32221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:20.431442 32221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:20.431855 32210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:20.432806 32210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:20.435252 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:20.426341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33741 (local address 127.0.0.1:41157)
0926 16:41:20.426464 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:20.426469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:20.426707 (+ 238us) server_negotiation.cc:407] Connection header received
0926 16:41:20.426892 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:20.426895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:20.426961 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:20.427015 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:20.428002 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:20.428835 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:20.429748 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:20.430086 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:20.430192 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:20.431871 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:20.431874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:20.431876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:20.431898 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:20.433481 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:20.433971 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:20.433977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:20.433979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:20.434055 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:20.434385 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:20.434390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:20.434392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:20.434574 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:20.434730 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:20.435003 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 16:41:20.435093 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:41:20.439424 32210 catalog_manager.cc:1347] Generated new cluster ID: f2548d5198b84b319e4ede23b6b228f5
I20260926 16:41:20.439503 32210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:20.445429 32210 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:20.446298 32210 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:20.453094 32210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57ae0b07fd854b8d9e9a7acee7cf1fc2: Generated new TSK 0
I20260926 16:41:20.453783 32210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:20.458689 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 32128
W20260926 16:41:20.463354 32113 connection.cc:535] client connection to 127.0.0.1:41157 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41157): Connection reset by peer (error 104)
2026-09-26T16:41:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:20 dist-test-slave-p39x krb5kdc[32227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:20 dist-test-slave-p39x krb5kdc[32227](info): set up 2 sockets
Sep 26 16:41:20 dist-test-slave-p39x krb5kdc[32227](info): commencing operation
krb5kdc: starting...
W20260926 16:41:21.497208 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:21 dist-test-slave-p39x krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:21.623819 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:21.738305 32240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:21.738605 32240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:21.738647 32240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:21.742935 32240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:21.743018 32240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:21.743038 32240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:21.743055 32240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:21.743072 32240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:21.748821 32240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35017
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:21.750010 32240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:21.751082 32240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:21.760140 32245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:21.760206 32246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:21.760397 32248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:21.760448 32240 server_base.cc:1034] running on GCE node
I20260926 16:41:21.761178 32240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:21.762342 32240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:21.763572 32240 hybrid_clock.cc:648] HybridClock initialized: now 1790440881763540 us; error 46 us; skew 500 ppm
Sep 26 16:41:21 dist-test-slave-p39x krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:21.766641 32240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:21.767625 32240 webserver.cc:460] Webserver started at http://127.0.0.1:39965/ using document root <none> and password file <none>
I20260926 16:41:21.768237 32240 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:21.768344 32240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:21.768618 32240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:21.770932 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "5af5b1c7b3d14d47bb01a43a02003f45"
format_stamp: "Formatted at 2026-09-26 16:41:21 on dist-test-slave-p39x"
server_key: "965583ff942c819c4696a44d1c000f02"
server_key_iv: "f85786e9ce0d27015ba317b5e289a070"
server_key_version: "encryptionkey@0"
I20260926 16:41:21.771518 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "5af5b1c7b3d14d47bb01a43a02003f45"
format_stamp: "Formatted at 2026-09-26 16:41:21 on dist-test-slave-p39x"
server_key: "965583ff942c819c4696a44d1c000f02"
server_key_iv: "f85786e9ce0d27015ba317b5e289a070"
server_key_version: "encryptionkey@0"
I20260926 16:41:21.775566 32240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260926 16:41:21.778313 32255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:21.779712 32240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 16:41:21.779878 32240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "5af5b1c7b3d14d47bb01a43a02003f45"
format_stamp: "Formatted at 2026-09-26 16:41:21 on dist-test-slave-p39x"
server_key: "965583ff942c819c4696a44d1c000f02"
server_key_iv: "f85786e9ce0d27015ba317b5e289a070"
server_key_version: "encryptionkey@0"
I20260926 16:41:21.780035 32240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:21.802702 32240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:21.806385 32240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:21.806627 32240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:21.814364 32240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35017
I20260926 16:41:21.814388 32307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35017 every 8 connection(s)
I20260926 16:41:21.815363 32240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:21.818305 32308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:21.821480 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 32240
I20260926 16:41:21.821630 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:21.821949 11898 external_mini_cluster.cc:1428] Setting key bc7fa9d5be06abb66cbc8e67362a2528
I20260926 16:41:21.824882 32308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45: Bootstrap starting.
I20260926 16:41:21.827183 32308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:21.827983 32308 log.cc:824] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:21.830437 32308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45: No bootstrap required, opened a new log
I20260926 16:41:21.831689 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:21.823513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42607 (local address 127.0.0.1:35017)
0926 16:41:21.824029 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:21.824044 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:21.824082 (+ 38us) server_negotiation.cc:407] Connection header received
0926 16:41:21.824863 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:21.824884 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:21.825301 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:21.825659 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:21.826085 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:21.827068 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:21.828028 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:21.828761 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:21.828939 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:21.829934 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:21.829939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:21.829955 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:21.830357 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:21.830416 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:21.830440 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:21.830611 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:21.830809 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 16:41:21.830958 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":158,"threads_started":1}
I20260926 16:41:21.834987 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:21.834906 32308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260926 16:41:21.835222 32308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:21.835279 32308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af5b1c7b3d14d47bb01a43a02003f45, State: Initialized, Role: FOLLOWER
I20260926 16:41:21.835688 32308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260926 16:41:21.835841 32308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:21.835937 32308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:21.836037 32308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:21.837278 32308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260926 16:41:21.837656 32308 leader_election.cc:302] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af5b1c7b3d14d47bb01a43a02003f45; no voters:
I20260926 16:41:21.838017 32308 leader_election.cc:288] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:21.838089 32318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:21.838337 32318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [term 1 LEADER]: Becoming Leader. State: Replica: 5af5b1c7b3d14d47bb01a43a02003f45, State: Running, Role: LEADER
I20260926 16:41:21.838711 32318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } }
I20260926 16:41:21.839418 32308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:21.840440 32320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af5b1c7b3d14d47bb01a43a02003f45. Latest consensus state: current_term: 1 leader_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } } }
I20260926 16:41:21.840579 32320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:21.840934 32319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af5b1c7b3d14d47bb01a43a02003f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35017 } } }
I20260926 16:41:21.841087 32319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:21.841876 32327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:21.842604 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:21.837339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36107 (local address 127.0.0.1:35017)
0926 16:41:21.837502 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:21.837507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:21.837695 (+ 188us) server_negotiation.cc:407] Connection header received
0926 16:41:21.837845 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:21.837854 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:21.837907 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:21.837994 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:21.838331 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:21.839067 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:21.840181 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:21.840569 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:21.840672 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:21.841861 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:21.841866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:21.841868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:21.841885 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:21.841915 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:21.841929 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:21.842022 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:21.842460 (+ 438us) server_negotiation.cc:299] Negotiation successful
0926 16:41:21.842506 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:41:21.845057 32327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:21.850942 32327 catalog_manager.cc:1347] Generated new cluster ID: fc76fc11242c46d89375f7a6775762e6
I20260926 16:41:21.851046 32327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:21.865355 32327 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:21.866662 32327 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:21.876380 32327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af5b1c7b3d14d47bb01a43a02003f45: Generated new TSK 0
I20260926 16:41:21.877326 32327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:21.887831 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 32240
W20260926 16:41:21.892818 32225 connection.cc:535] client connection to 127.0.0.1:35017 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35017): Connection reset by peer (error 104)
2026-09-26T16:41:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:21 dist-test-slave-p39x krb5kdc[32339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:21 dist-test-slave-p39x krb5kdc[32339](info): set up 2 sockets
Sep 26 16:41:21 dist-test-slave-p39x krb5kdc[32339](info): commencing operation
krb5kdc: starting...
W20260926 16:41:22.929239 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:23 dist-test-slave-p39x krb5kdc[32339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:23.056769 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:23.168149 32352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:23.168457 32352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:23.168504 32352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:23.172466 32352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:23.172523 32352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:23.172544 32352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:23.172559 32352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:23.172575 32352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:23.177943 32352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40027
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:23.179005 32352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:23.180019 32352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:23.188683 32358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:23.188683 32360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:23.188717 32357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:23.189147 32352 server_base.cc:1034] running on GCE node
I20260926 16:41:23.189769 32352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:23.190922 32352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:23.192138 32352 hybrid_clock.cc:648] HybridClock initialized: now 1790440883192125 us; error 27 us; skew 500 ppm
Sep 26 16:41:23 dist-test-slave-p39x krb5kdc[32339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:23.195158 32352 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:23.196084 32352 webserver.cc:460] Webserver started at http://127.0.0.1:36483/ using document root <none> and password file <none>
I20260926 16:41:23.196724 32352 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:23.196816 32352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:23.197059 32352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:23.199154 32352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "5270dca3276c4bf9b65fdab5b8cfa33e"
format_stamp: "Formatted at 2026-09-26 16:41:23 on dist-test-slave-p39x"
server_key: "cf6c810f39c6c6bb178dd570b849124d"
server_key_iv: "939d195ad3846e94e0d6bb1cd39958f2"
server_key_version: "encryptionkey@0"
I20260926 16:41:23.199745 32352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "5270dca3276c4bf9b65fdab5b8cfa33e"
format_stamp: "Formatted at 2026-09-26 16:41:23 on dist-test-slave-p39x"
server_key: "cf6c810f39c6c6bb178dd570b849124d"
server_key_iv: "939d195ad3846e94e0d6bb1cd39958f2"
server_key_version: "encryptionkey@0"
I20260926 16:41:23.203706 32352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:41:23.206668 32367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:23.207989 32352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:41:23.208161 32352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "5270dca3276c4bf9b65fdab5b8cfa33e"
format_stamp: "Formatted at 2026-09-26 16:41:23 on dist-test-slave-p39x"
server_key: "cf6c810f39c6c6bb178dd570b849124d"
server_key_iv: "939d195ad3846e94e0d6bb1cd39958f2"
server_key_version: "encryptionkey@0"
I20260926 16:41:23.208297 32352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:23.213057 32352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:23.216388 32352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:23.216595 32352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:23.223448 32352 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40027
I20260926 16:41:23.223515 32419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40027 every 8 connection(s)
I20260926 16:41:23.224530 32352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:23.227267 32420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:23.232787 32420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e: Bootstrap starting.
I20260926 16:41:23.233203 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 32352
I20260926 16:41:23.233353 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:23.233639 11898 external_mini_cluster.cc:1428] Setting key e546ab2513ecec913da7ff5a92633867
I20260926 16:41:23.235459 32420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:23.236316 32420 log.cc:824] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:23.238478 32420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e: No bootstrap required, opened a new log
I20260926 16:41:23.239194 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:23.234980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54125 (local address 127.0.0.1:40027)
0926 16:41:23.235432 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:23.235452 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:23.235494 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:41:23.236419 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:23.236444 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:23.236842 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:23.237198 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:23.237986 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:23.237990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:23.238001 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:23.238011 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:23.238063 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:23.238082 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:23.238196 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:23.238294 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 16:41:23.238454 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":110,"threads_started":1}
I20260926 16:41:23.242062 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:23.242441 32420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40027 } }
I20260926 16:41:23.242672 32420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:23.242745 32420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5270dca3276c4bf9b65fdab5b8cfa33e, State: Initialized, Role: FOLLOWER
I20260926 16:41:23.243161 32420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40027 } }
I20260926 16:41:23.243325 32420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:23.243413 32420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:23.243567 32420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:23.244776 32420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40027 } }
I20260926 16:41:23.245194 32420 leader_election.cc:302] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5270dca3276c4bf9b65fdab5b8cfa33e; no voters:
I20260926 16:41:23.245600 32420 leader_election.cc:288] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:23.245657 32430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:23.245886 32430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [term 1 LEADER]: Becoming Leader. State: Replica: 5270dca3276c4bf9b65fdab5b8cfa33e, State: Running, Role: LEADER
I20260926 16:41:23.245923 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:23.244303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60375 (local address 127.0.0.1:40027)
0926 16:41:23.244440 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:23.244446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:23.244597 (+ 151us) server_negotiation.cc:407] Connection header received
0926 16:41:23.244730 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:23.244732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:23.244763 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:23.244825 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:23.245536 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:23.245538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:23.245540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:23.245550 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:23.245575 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:23.245585 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:23.245684 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:23.245762 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 16:41:23.245802 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:41:23.246194 32430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40027 } }
I20260926 16:41:23.246992 32420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:23.247571 32432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40027 } } }
I20260926 16:41:23.247697 32432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:23.247979 32431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5270dca3276c4bf9b65fdab5b8cfa33e. Latest consensus state: current_term: 1 leader_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5270dca3276c4bf9b65fdab5b8cfa33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40027 } } }
I20260926 16:41:23.248092 32431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:23.248158 32436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:23.251653 32436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:23.257314 32436 catalog_manager.cc:1347] Generated new cluster ID: 0515436d8a1a4145b79bb09d2c413793
I20260926 16:41:23.257396 32436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:23.266875 32436 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:23.267220 32436 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:23.270977 32436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5270dca3276c4bf9b65fdab5b8cfa33e: Generated new TSK 0
I20260926 16:41:23.271745 32436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:23.284790 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 32352
2026-09-26T16:41:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:23 dist-test-slave-p39x krb5kdc[32451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:23 dist-test-slave-p39x krb5kdc[32451](info): set up 2 sockets
Sep 26 16:41:23 dist-test-slave-p39x krb5kdc[32451](info): commencing operation
krb5kdc: starting...
W20260926 16:41:24.337257 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:24 dist-test-slave-p39x krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:24.468317 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:24.582727 32464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:24.583015 32464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:24.583060 32464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:24.587051 32464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:24.587121 32464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:24.587141 32464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:24.587158 32464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:24.587173 32464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:24.592553 32464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:24.593623 32464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:24.594635 32464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:24.603904 32472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:24.604039 32470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:24.603904 32469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:24.604148 32464 server_base.cc:1034] running on GCE node
I20260926 16:41:24.604910 32464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:24.606099 32464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:24.607326 32464 hybrid_clock.cc:648] HybridClock initialized: now 1790440884607296 us; error 52 us; skew 500 ppm
Sep 26 16:41:24 dist-test-slave-p39x krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:24.610428 32464 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:24.611436 32464 webserver.cc:460] Webserver started at http://127.0.0.1:33701/ using document root <none> and password file <none>
I20260926 16:41:24.612076 32464 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:24.612175 32464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:24.612488 32464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:24.615234 32464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "c340551dd536401eac8f4ab94974a3ad"
format_stamp: "Formatted at 2026-09-26 16:41:24 on dist-test-slave-p39x"
server_key: "2c3728b506caa38e679ad25b8e2fc555"
server_key_iv: "4c1c08aa2b3ca5bae5ea2c73f56ffee2"
server_key_version: "encryptionkey@0"
I20260926 16:41:24.615873 32464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "c340551dd536401eac8f4ab94974a3ad"
format_stamp: "Formatted at 2026-09-26 16:41:24 on dist-test-slave-p39x"
server_key: "2c3728b506caa38e679ad25b8e2fc555"
server_key_iv: "4c1c08aa2b3ca5bae5ea2c73f56ffee2"
server_key_version: "encryptionkey@0"
I20260926 16:41:24.619915 32464 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:41:24.622639 32479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:24.623916 32464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:41:24.624069 32464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "c340551dd536401eac8f4ab94974a3ad"
format_stamp: "Formatted at 2026-09-26 16:41:24 on dist-test-slave-p39x"
server_key: "2c3728b506caa38e679ad25b8e2fc555"
server_key_iv: "4c1c08aa2b3ca5bae5ea2c73f56ffee2"
server_key_version: "encryptionkey@0"
I20260926 16:41:24.624190 32464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:24.640811 32464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:24.643898 32464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:24.644055 32464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:24.651168 32464 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41549
I20260926 16:41:24.651257 32531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41549 every 8 connection(s)
I20260926 16:41:24.652204 32464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:24.655071 32532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:24.655092 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 32464
I20260926 16:41:24.655231 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:24.655514 11898 external_mini_cluster.cc:1428] Setting key 061d029f2ce089a44db0f871a405ef7f
I20260926 16:41:24.662343 32532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad: Bootstrap starting.
I20260926 16:41:24.665112 32532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad: Neither blocks nor log segments found. Creating new log.
Sep 26 16:41:24 dist-test-slave-p39x krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:41:24.665972 32532 log.cc:824] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:24.668336 32532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad: No bootstrap required, opened a new log
I20260926 16:41:24.671053 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:24.656965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43787 (local address 127.0.0.1:41549)
0926 16:41:24.657397 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:24.657417 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:24.657460 (+ 43us) server_negotiation.cc:407] Connection header received
0926 16:41:24.658501 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:24.658526 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:24.658968 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:24.659337 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:24.659982 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:24.661511 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:24.662364 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:24.663039 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:24.663189 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:24.665882 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:24.665889 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:24.665905 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:24.665945 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:24.667912 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:24.668351 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:24.668360 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:24.668367 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:24.668508 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:24.668778 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:24.668781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:24.668783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:24.669281 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:24.669462 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:24.669679 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 16:41:24.669986 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
I20260926 16:41:24.673110 32532 raft_consensus.cc:348] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c340551dd536401eac8f4ab94974a3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260926 16:41:24.673358 32532 raft_consensus.cc:374] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:24.673441 32532 raft_consensus.cc:729] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c340551dd536401eac8f4ab94974a3ad, State: Initialized, Role: FOLLOWER
I20260926 16:41:24.673918 32532 consensus_queue.cc:260] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c340551dd536401eac8f4ab94974a3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260926 16:41:24.674042 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:24.674083 32532 raft_consensus.cc:388] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:24.674156 32532 raft_consensus.cc:482] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:24.674250 32532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:24.675199 32532 raft_consensus.cc:504] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c340551dd536401eac8f4ab94974a3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260926 16:41:24.675616 32532 leader_election.cc:302] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c340551dd536401eac8f4ab94974a3ad; no voters:
I20260926 16:41:24.675968 32532 leader_election.cc:288] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:24.676088 32541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:24.676326 32541 raft_consensus.cc:686] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [term 1 LEADER]: Becoming Leader. State: Replica: c340551dd536401eac8f4ab94974a3ad, State: Running, Role: LEADER
I20260926 16:41:24.676760 32541 consensus_queue.cc:237] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c340551dd536401eac8f4ab94974a3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } }
I20260926 16:41:24.677369 32532 sys_catalog.cc:564] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:24.678542 32544 sys_catalog.cc:455] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader c340551dd536401eac8f4ab94974a3ad. Latest consensus state: current_term: 1 leader_uuid: "c340551dd536401eac8f4ab94974a3ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c340551dd536401eac8f4ab94974a3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } } }
I20260926 16:41:24.678656 32544 sys_catalog.cc:458] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:24.678995 32543 sys_catalog.cc:455] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c340551dd536401eac8f4ab94974a3ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c340551dd536401eac8f4ab94974a3ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41549 } } }
I20260926 16:41:24.679244 32543 sys_catalog.cc:458] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad [sys.catalog]: This master's current role is: LEADER
W20260926 16:41:24.682281 32557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:24.682364 32557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:24.682473 32551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:24.683363 32551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:24.685714 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:24.676354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36105 (local address 127.0.0.1:41549)
0926 16:41:24.676463 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:24.676468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:24.676705 (+ 237us) server_negotiation.cc:407] Connection header received
0926 16:41:24.676809 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:24.676812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:24.676864 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:24.676955 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:24.677839 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:24.678723 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:24.679947 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:24.680195 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:24.680311 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:24.681923 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:24.681927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:24.681930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:24.681968 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:24.683701 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:24.684230 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:24.684234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:24.684236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:24.684322 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:24.684691 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:24.684695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:24.684697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:24.684929 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:24.685045 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:24.685433 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 16:41:24.685563 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:41:24.689273 32551 catalog_manager.cc:1347] Generated new cluster ID: dd0305c935eb41abb2a33a1fe816c584
I20260926 16:41:24.689416 32551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:24.720772 32551 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:24.721753 32551 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:24.729161 32551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c340551dd536401eac8f4ab94974a3ad: Generated new TSK 0
I20260926 16:41:24.729964 32551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:24.742084 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 32464
2026-09-26T16:41:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:24 dist-test-slave-p39x krb5kdc[32563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:24 dist-test-slave-p39x krb5kdc[32563](info): set up 2 sockets
Sep 26 16:41:24 dist-test-slave-p39x krb5kdc[32563](info): commencing operation
krb5kdc: starting...
W20260926 16:41:25.781122 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:25 dist-test-slave-p39x krb5kdc[32563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:25.910339 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:26.025267 32576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:26.025596 32576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:26.025678 32576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:26.029765 32576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:26.029870 32576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:26.029906 32576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:26.029966 32576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:26.030005 32576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:26.035480 32576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:26.036710 32576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:26.037770 32576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:26.048976 32581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:26.048978 32584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:26.049023 32582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:26.049368 32576 server_base.cc:1034] running on GCE node
I20260926 16:41:26.049978 32576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:26.051239 32576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:26.052434 32576 hybrid_clock.cc:648] HybridClock initialized: now 1790440886052414 us; error 28 us; skew 500 ppm
Sep 26 16:41:26 dist-test-slave-p39x krb5kdc[32563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440886, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:26.059689 32576 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:26.060761 32576 webserver.cc:460] Webserver started at http://127.0.0.1:37465/ using document root <none> and password file <none>
I20260926 16:41:26.061365 32576 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:26.061429 32576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:26.061602 32576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:26.063773 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "2c170880760944e88ee7e51c2dfc3da1"
format_stamp: "Formatted at 2026-09-26 16:41:26 on dist-test-slave-p39x"
server_key: "f9259bca406ad4bb0f2a00351ce91ef9"
server_key_iv: "82de7c071a09406a3aef457bfb73cfc5"
server_key_version: "encryptionkey@0"
I20260926 16:41:26.064406 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "2c170880760944e88ee7e51c2dfc3da1"
format_stamp: "Formatted at 2026-09-26 16:41:26 on dist-test-slave-p39x"
server_key: "f9259bca406ad4bb0f2a00351ce91ef9"
server_key_iv: "82de7c071a09406a3aef457bfb73cfc5"
server_key_version: "encryptionkey@0"
I20260926 16:41:26.068506 32576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:41:26.071311 32591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:26.072844 32576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:26.072968 32576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "2c170880760944e88ee7e51c2dfc3da1"
format_stamp: "Formatted at 2026-09-26 16:41:26 on dist-test-slave-p39x"
server_key: "f9259bca406ad4bb0f2a00351ce91ef9"
server_key_iv: "82de7c071a09406a3aef457bfb73cfc5"
server_key_version: "encryptionkey@0"
I20260926 16:41:26.073074 32576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:26.106480 32576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:26.109968 32576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:26.110133 32576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:26.117720 32576 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42123
I20260926 16:41:26.117745 32643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42123 every 8 connection(s)
I20260926 16:41:26.118744 32576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:26.121563 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:26.127588 32644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1: Bootstrap starting.
I20260926 16:41:26.127818 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 32576
I20260926 16:41:26.127995 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:26.128274 11898 external_mini_cluster.cc:1428] Setting key d30fb1e06a40fe9125002a1f36c334d3
I20260926 16:41:26.130509 32644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:26.131251 32644 log.cc:824] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:26.133610 32644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1: No bootstrap required, opened a new log
I20260926 16:41:26.138175 32647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:26.129804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49945 (local address 127.0.0.1:42123)
0926 16:41:26.130294 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:26.130332 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:26.130368 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:41:26.131104 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:26.131122 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:26.131542 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:26.131899 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:26.132435 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:26.133417 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:26.134685 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:26.135188 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:26.135324 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:26.136258 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:26.136263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:26.136278 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:26.136784 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:26.136844 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:26.136868 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:26.136946 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:26.137113 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 16:41:26.137357 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":169,"threads_started":1}
I20260926 16:41:26.138104 32644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c170880760944e88ee7e51c2dfc3da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 16:41:26.138411 32644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:26.138482 32644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c170880760944e88ee7e51c2dfc3da1, State: Initialized, Role: FOLLOWER
I20260926 16:41:26.138929 32644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c170880760944e88ee7e51c2dfc3da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 16:41:26.139066 32644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:26.139206 32644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:26.139328 32644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:26.140632 32644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c170880760944e88ee7e51c2dfc3da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 16:41:26.141028 32644 leader_election.cc:302] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c170880760944e88ee7e51c2dfc3da1; no voters:
I20260926 16:41:26.141423 32644 leader_election.cc:288] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:26.141534 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:26.141604 32649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:26.141842 32649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [term 1 LEADER]: Becoming Leader. State: Replica: 2c170880760944e88ee7e51c2dfc3da1, State: Running, Role: LEADER
I20260926 16:41:26.142190 32649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c170880760944e88ee7e51c2dfc3da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } }
I20260926 16:41:26.142828 32644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:26.143371 32649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c170880760944e88ee7e51c2dfc3da1. Latest consensus state: current_term: 1 leader_uuid: "2c170880760944e88ee7e51c2dfc3da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c170880760944e88ee7e51c2dfc3da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } } }
I20260926 16:41:26.143553 32649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:26.143862 32651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c170880760944e88ee7e51c2dfc3da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c170880760944e88ee7e51c2dfc3da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42123 } } }
I20260926 16:41:26.143980 32651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:26.144009 32660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:26.147703 32660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:26.149617 32647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:26.144396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33019 (local address 127.0.0.1:42123)
0926 16:41:26.144515 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:26.144519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:26.144745 (+ 226us) server_negotiation.cc:407] Connection header received
0926 16:41:26.144940 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:26.144943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:26.145006 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:26.145063 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:26.145595 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:26.146368 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:26.147512 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:26.147855 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:26.147956 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:26.148987 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:26.148991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:26.148994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:26.149008 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:26.149038 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:26.149051 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:26.149110 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:26.149427 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 16:41:26.149482 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:41:26.154452 32660 catalog_manager.cc:1347] Generated new cluster ID: d67dca8f320d4a27aebb85335275aaaf
I20260926 16:41:26.154541 32660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:26.180493 32660 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:26.181702 32660 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:26.191917 32660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c170880760944e88ee7e51c2dfc3da1: Generated new TSK 0
I20260926 16:41:26.192992 32660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:26.203312 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 32576
2026-09-26T16:41:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:26 dist-test-slave-p39x krb5kdc[32675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:26 dist-test-slave-p39x krb5kdc[32675](info): set up 2 sockets
Sep 26 16:41:26 dist-test-slave-p39x krb5kdc[32675](info): commencing operation
krb5kdc: starting...
W20260926 16:41:27.325189 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:27 dist-test-slave-p39x krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:27.444975 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:27.557502 32688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:27.557853 32688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:27.557930 32688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:27.562204 32688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:27.562325 32688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:27.562367 32688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:27.562407 32688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:27.562443 32688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:27.567965 32688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:27.569187 32688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:27.570286 32688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:27.579293 32696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:27.579470 32688 server_base.cc:1034] running on GCE node
W20260926 16:41:27.579293 32693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:27.579370 32694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:27.580168 32688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:27.581398 32688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:27.582620 32688 hybrid_clock.cc:648] HybridClock initialized: now 1790440887582591 us; error 40 us; skew 500 ppm
Sep 26 16:41:27 dist-test-slave-p39x krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:27.585412 32688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:27.586301 32688 webserver.cc:460] Webserver started at http://127.0.0.1:45591/ using document root <none> and password file <none>
I20260926 16:41:27.586876 32688 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:27.586966 32688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:27.587201 32688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:27.589313 32688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "d012b791ef6444599329afc236f9065d"
format_stamp: "Formatted at 2026-09-26 16:41:27 on dist-test-slave-p39x"
server_key: "5e9c3d06714288df538466bd29e36cb4"
server_key_iv: "81d85f1b6517fbd406db9050011b5de2"
server_key_version: "encryptionkey@0"
I20260926 16:41:27.589875 32688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "d012b791ef6444599329afc236f9065d"
format_stamp: "Formatted at 2026-09-26 16:41:27 on dist-test-slave-p39x"
server_key: "5e9c3d06714288df538466bd29e36cb4"
server_key_iv: "81d85f1b6517fbd406db9050011b5de2"
server_key_version: "encryptionkey@0"
I20260926 16:41:27.593721 32688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:41:27.596153 32703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:27.597441 32688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:27.597611 32688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "d012b791ef6444599329afc236f9065d"
format_stamp: "Formatted at 2026-09-26 16:41:27 on dist-test-slave-p39x"
server_key: "5e9c3d06714288df538466bd29e36cb4"
server_key_iv: "81d85f1b6517fbd406db9050011b5de2"
server_key_version: "encryptionkey@0"
I20260926 16:41:27.597729 32688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:27.602371 32688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:27.605516 32688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:27.605711 32688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:27.612762 32688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34561
I20260926 16:41:27.612803 32755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34561 every 8 connection(s)
I20260926 16:41:27.613725 32688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:27.616699 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:27.621177 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 32688
I20260926 16:41:27.621345 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:27.621618 11898 external_mini_cluster.cc:1428] Setting key 74b6172c5b68a2f579ae4c9703c9469e
I20260926 16:41:27.622864 32756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d: Bootstrap starting.
I20260926 16:41:27.625428 32756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:27.626320 32756 log.cc:824] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:27.626950 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:27.622969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42153 (local address 127.0.0.1:34561)
0926 16:41:27.623402 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:27.623418 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:27.623460 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:41:27.624134 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:27.624151 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:27.624529 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:27.624812 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:27.625695 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:27.625700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:27.625711 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:27.625722 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:27.625776 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:27.625795 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:27.625881 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:27.626016 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 16:41:27.626184 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
I20260926 16:41:27.628407 32756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d: No bootstrap required, opened a new log
I20260926 16:41:27.629887 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:27.633209 32756 raft_consensus.cc:348] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d012b791ef6444599329afc236f9065d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34561 } }
I20260926 16:41:27.633447 32756 raft_consensus.cc:374] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:27.633507 32756 raft_consensus.cc:729] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d012b791ef6444599329afc236f9065d, State: Initialized, Role: FOLLOWER
I20260926 16:41:27.634016 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:27.632218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37303 (local address 127.0.0.1:34561)
0926 16:41:27.632385 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:27.632390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:27.632617 (+ 227us) server_negotiation.cc:407] Connection header received
0926 16:41:27.632699 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:27.632702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:27.632738 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:27.632834 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:27.633612 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:27.633616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:27.633619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:27.633632 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:27.633661 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:27.633675 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:27.633768 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:27.633861 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 16:41:27.633899 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:41:27.633993 32756 consensus_queue.cc:260] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d012b791ef6444599329afc236f9065d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34561 } }
I20260926 16:41:27.634147 32756 raft_consensus.cc:388] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:27.634215 32756 raft_consensus.cc:482] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:27.634344 32756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:27.635542 32756 raft_consensus.cc:504] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d012b791ef6444599329afc236f9065d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34561 } }
I20260926 16:41:27.635903 32756 leader_election.cc:302] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d012b791ef6444599329afc236f9065d; no voters:
I20260926 16:41:27.636230 32756 leader_election.cc:288] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:27.636389 32766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:27.636644 32766 raft_consensus.cc:686] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [term 1 LEADER]: Becoming Leader. State: Replica: d012b791ef6444599329afc236f9065d, State: Running, Role: LEADER
I20260926 16:41:27.637028 32766 consensus_queue.cc:237] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d012b791ef6444599329afc236f9065d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34561 } }
I20260926 16:41:27.637501 32756 sys_catalog.cc:564] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:27.638789 300 sys_catalog.cc:455] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d012b791ef6444599329afc236f9065d. Latest consensus state: current_term: 1 leader_uuid: "d012b791ef6444599329afc236f9065d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d012b791ef6444599329afc236f9065d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34561 } } }
I20260926 16:41:27.638779 32767 sys_catalog.cc:455] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d012b791ef6444599329afc236f9065d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d012b791ef6444599329afc236f9065d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34561 } } }
I20260926 16:41:27.638952 300 sys_catalog.cc:458] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:27.639024 32767 sys_catalog.cc:458] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:27.639387 310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:27.642416 310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:27.648186 310 catalog_manager.cc:1347] Generated new cluster ID: e03540fb081f4f299ec3f0e293f17c2e
I20260926 16:41:27.648270 310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:27.677453 310 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:27.677860 310 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:27.687500 310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d012b791ef6444599329afc236f9065d: Generated new TSK 0
I20260926 16:41:27.688460 310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:27.697204 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 32688
2026-09-26T16:41:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:27 dist-test-slave-p39x krb5kdc[319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:27 dist-test-slave-p39x krb5kdc[319](info): set up 2 sockets
Sep 26 16:41:27 dist-test-slave-p39x krb5kdc[319](info): commencing operation
krb5kdc: starting...
W20260926 16:41:28.749312 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:28 dist-test-slave-p39x krb5kdc[319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:28.881589 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:29.000860 332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:29.001188 332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:29.001236 332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:29.005707 332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:29.005800 332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:29.005821 332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:29.005839 332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:29.005856 332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:29.012035 332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:29.013335 332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:29.014454 332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:29.023682 338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:29.023864 337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:29.023880 340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:29.024123 332 server_base.cc:1034] running on GCE node
I20260926 16:41:29.024783 332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:29.026027 332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:29.027330 332 hybrid_clock.cc:648] HybridClock initialized: now 1790440889027296 us; error 47 us; skew 500 ppm
Sep 26 16:41:29 dist-test-slave-p39x krb5kdc[319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:29.030766 332 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:29.031860 332 webserver.cc:460] Webserver started at http://127.0.0.1:40475/ using document root <none> and password file <none>
I20260926 16:41:29.032605 332 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:29.032713 332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:29.032972 332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:29.035248 332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "b65cec805f9745c7a0ee66356ba18b57"
format_stamp: "Formatted at 2026-09-26 16:41:29 on dist-test-slave-p39x"
server_key: "3d23e90a11924c7dd6b1f447ddfe366b"
server_key_iv: "b7a593f57acca0094cfad3193855cdc3"
server_key_version: "encryptionkey@0"
I20260926 16:41:29.036067 332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "b65cec805f9745c7a0ee66356ba18b57"
format_stamp: "Formatted at 2026-09-26 16:41:29 on dist-test-slave-p39x"
server_key: "3d23e90a11924c7dd6b1f447ddfe366b"
server_key_iv: "b7a593f57acca0094cfad3193855cdc3"
server_key_version: "encryptionkey@0"
I20260926 16:41:29.040563 332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:41:29.043429 347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:29.045298 332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:29.045472 332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "b65cec805f9745c7a0ee66356ba18b57"
format_stamp: "Formatted at 2026-09-26 16:41:29 on dist-test-slave-p39x"
server_key: "3d23e90a11924c7dd6b1f447ddfe366b"
server_key_iv: "b7a593f57acca0094cfad3193855cdc3"
server_key_version: "encryptionkey@0"
I20260926 16:41:29.045621 332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:29.074054 332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:29.077550 332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:29.077718 332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:29.085036 332 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45921
I20260926 16:41:29.085062 399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45921 every 8 connection(s)
I20260926 16:41:29.086066 332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:29.088969 400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:29.089823 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 332
I20260926 16:41:29.089967 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:29.090284 11898 external_mini_cluster.cc:1428] Setting key 1709c3203bb86657fc9bde6df7d41c41
I20260926 16:41:29.095103 400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57: Bootstrap starting.
I20260926 16:41:29.097375 400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:29.098107 400 log.cc:824] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:41:29 dist-test-slave-p39x krb5kdc[319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:41:29.100168 400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57: No bootstrap required, opened a new log
I20260926 16:41:29.106410 403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:29.091746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:51895 (local address 127.0.0.1:45921)
0926 16:41:29.092194 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:29.092209 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:29.092243 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:41:29.092951 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:29.092968 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:29.093303 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:29.093643 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:29.094450 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:29.095481 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:29.096379 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:29.096852 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:29.097026 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:29.099832 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:29.099837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:29.099850 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:29.099876 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:29.102803 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:29.103418 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:29.103424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:29.103429 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:29.103555 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:29.103926 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:29.103931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:29.103938 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:29.104461 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:29.104694 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:29.105047 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 16:41:29.105297 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":136,"threads_started":1}
I20260926 16:41:29.106307 400 raft_consensus.cc:348] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65cec805f9745c7a0ee66356ba18b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45921 } }
I20260926 16:41:29.106582 400 raft_consensus.cc:374] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:29.106634 400 raft_consensus.cc:729] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b65cec805f9745c7a0ee66356ba18b57, State: Initialized, Role: FOLLOWER
I20260926 16:41:29.107147 400 consensus_queue.cc:260] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65cec805f9745c7a0ee66356ba18b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45921 } }
I20260926 16:41:29.107299 400 raft_consensus.cc:388] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:29.107343 400 raft_consensus.cc:482] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:29.107411 400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:29.108700 400 raft_consensus.cc:504] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65cec805f9745c7a0ee66356ba18b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45921 } }
I20260926 16:41:29.109097 400 leader_election.cc:302] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b65cec805f9745c7a0ee66356ba18b57; no voters:
I20260926 16:41:29.109470 400 leader_election.cc:288] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:29.109566 405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:29.109755 405 raft_consensus.cc:686] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [term 1 LEADER]: Becoming Leader. State: Replica: b65cec805f9745c7a0ee66356ba18b57, State: Running, Role: LEADER
I20260926 16:41:29.110128 405 consensus_queue.cc:237] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65cec805f9745c7a0ee66356ba18b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45921 } }
I20260926 16:41:29.110634 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:29.110831 400 sys_catalog.cc:564] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:29.111806 406 sys_catalog.cc:455] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b65cec805f9745c7a0ee66356ba18b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65cec805f9745c7a0ee66356ba18b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45921 } } }
I20260926 16:41:29.111955 406 sys_catalog.cc:458] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:29.112440 407 sys_catalog.cc:455] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b65cec805f9745c7a0ee66356ba18b57. Latest consensus state: current_term: 1 leader_uuid: "b65cec805f9745c7a0ee66356ba18b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65cec805f9745c7a0ee66356ba18b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45921 } } }
I20260926 16:41:29.112648 407 sys_catalog.cc:458] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57 [sys.catalog]: This master's current role is: LEADER
W20260926 16:41:29.116813 426 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:29.116931 426 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:29.117043 419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:29.117862 419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:29.124053 403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:29.114414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:49029 (local address 127.0.0.1:45921)
0926 16:41:29.114549 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:29.114554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:29.114684 (+ 130us) server_negotiation.cc:407] Connection header received
0926 16:41:29.114812 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:29.114815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:29.114871 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:29.115003 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:29.115863 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:29.116656 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:29.117821 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:29.118157 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:29.118294 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:29.120048 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:29.120053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:29.120056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:29.120085 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:29.121982 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:29.122592 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:29.122597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:29.122599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:29.122699 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:29.123083 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:29.123086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:29.123088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:29.123293 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:29.123462 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:29.123764 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 16:41:29.123876 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:41:29.124439 419 catalog_manager.cc:1347] Generated new cluster ID: a5a6fcfd6ffa4e3383895070ec64fac9
I20260926 16:41:29.124509 419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:29.151105 419 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:29.152201 419 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:29.160399 419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b65cec805f9745c7a0ee66356ba18b57: Generated new TSK 0
I20260926 16:41:29.161237 419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:29.169920 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 332
W20260926 16:41:29.175522 317 connection.cc:535] client connection to 127.0.0.1:45921 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45921): Connection reset by peer (error 104)
2026-09-26T16:41:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:29 dist-test-slave-p39x krb5kdc[432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:29 dist-test-slave-p39x krb5kdc[432](info): set up 2 sockets
Sep 26 16:41:29 dist-test-slave-p39x krb5kdc[432](info): commencing operation
krb5kdc: starting...
W20260926 16:41:30.225195 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:30 dist-test-slave-p39x krb5kdc[432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:30.352308 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:30.465377 445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:30.465728 445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:30.465817 445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:30.469939 445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:30.470052 445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:30.470091 445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:30.470130 445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:30.470168 445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:30.475724 445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:30.476944 445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:30.478031 445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:30.487308 445 server_base.cc:1034] running on GCE node
W20260926 16:41:30.487234 451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:30.487246 453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:30.487234 450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:30.488193 445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:30.489509 445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:30.490746 445 hybrid_clock.cc:648] HybridClock initialized: now 1790440890490681 us; error 70 us; skew 500 ppm
Sep 26 16:41:30 dist-test-slave-p39x krb5kdc[432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440890, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:30.493750 445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:30.494679 445 webserver.cc:460] Webserver started at http://127.0.0.1:37413/ using document root <none> and password file <none>
I20260926 16:41:30.495231 445 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:30.495291 445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:30.495457 445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:30.497648 445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "ad668ccfa0fb40eb83a64de7a2e803da"
format_stamp: "Formatted at 2026-09-26 16:41:30 on dist-test-slave-p39x"
server_key: "0d3b91daf6afdf5ed824c47658345956"
server_key_iv: "eab7b67a87b52d75a83d081420f567a6"
server_key_version: "encryptionkey@0"
I20260926 16:41:30.498198 445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "ad668ccfa0fb40eb83a64de7a2e803da"
format_stamp: "Formatted at 2026-09-26 16:41:30 on dist-test-slave-p39x"
server_key: "0d3b91daf6afdf5ed824c47658345956"
server_key_iv: "eab7b67a87b52d75a83d081420f567a6"
server_key_version: "encryptionkey@0"
I20260926 16:41:30.503088 445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 16:41:30.506985 460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:30.508662 445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:30.508837 445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "ad668ccfa0fb40eb83a64de7a2e803da"
format_stamp: "Formatted at 2026-09-26 16:41:30 on dist-test-slave-p39x"
server_key: "0d3b91daf6afdf5ed824c47658345956"
server_key_iv: "eab7b67a87b52d75a83d081420f567a6"
server_key_version: "encryptionkey@0"
I20260926 16:41:30.508987 445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:30.521855 445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:30.525131 445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:30.525350 445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:30.532845 445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40377
I20260926 16:41:30.532923 512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40377 every 8 connection(s)
I20260926 16:41:30.533926 445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:30.536935 513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:30.539179 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 445
I20260926 16:41:30.539350 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:30.539665 11898 external_mini_cluster.cc:1428] Setting key 2711bbf0dc85f574f20eee5c721e737c
I20260926 16:41:30.543120 513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da: Bootstrap starting.
I20260926 16:41:30.545745 513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:30.546728 513 log.cc:824] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:30.548803 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:30.541036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:34845 (local address 127.0.0.1:40377)
0926 16:41:30.541442 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:30.541464 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:30.541499 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:41:30.542152 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:30.542170 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:30.542516 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:30.542796 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:30.543201 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:30.544212 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:30.545453 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:30.545980 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:30.546102 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:30.547050 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:30.547055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:30.547068 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:30.547453 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:30.547508 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:30.547529 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:30.547630 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:30.547822 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 16:41:30.548004 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":131,"threads_started":1}
I20260926 16:41:30.548986 513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da: No bootstrap required, opened a new log
I20260926 16:41:30.551988 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:30.553081 513 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40377 } }
I20260926 16:41:30.553397 513 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:30.553462 513 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad668ccfa0fb40eb83a64de7a2e803da, State: Initialized, Role: FOLLOWER
I20260926 16:41:30.553980 513 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40377 } }
I20260926 16:41:30.554163 513 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:30.554298 513 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:30.554427 513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:30.555401 513 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40377 } }
I20260926 16:41:30.555755 513 leader_election.cc:302] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad668ccfa0fb40eb83a64de7a2e803da; no voters:
I20260926 16:41:30.556097 513 leader_election.cc:288] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:30.556180 523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:30.556383 523 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [term 1 LEADER]: Becoming Leader. State: Replica: ad668ccfa0fb40eb83a64de7a2e803da, State: Running, Role: LEADER
I20260926 16:41:30.556732 523 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40377 } }
I20260926 16:41:30.557462 513 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:30.558490 524 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40377 } } }
I20260926 16:41:30.558625 524 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:30.559013 532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:30.559360 525 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad668ccfa0fb40eb83a64de7a2e803da. Latest consensus state: current_term: 1 leader_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad668ccfa0fb40eb83a64de7a2e803da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40377 } } }
I20260926 16:41:30.560143 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:30.554138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:55511 (local address 127.0.0.1:40377)
0926 16:41:30.554260 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:30.554265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:30.554469 (+ 204us) server_negotiation.cc:407] Connection header received
0926 16:41:30.554596 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:30.554599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:30.554655 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:30.554743 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:30.555025 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:30.555610 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:30.556751 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:30.557051 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:30.557111 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:30.559148 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:30.559152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:30.559155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:30.559170 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:30.559204 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:30.559217 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:30.559274 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:30.559920 (+ 646us) server_negotiation.cc:299] Negotiation successful
0926 16:41:30.559963 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:41:30.560156 525 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:30.562422 532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:30.568202 532 catalog_manager.cc:1347] Generated new cluster ID: abc4f6216d624e83adcd0a09bcc83525
I20260926 16:41:30.568288 532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:30.579041 532 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:30.580020 532 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:30.588869 532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad668ccfa0fb40eb83a64de7a2e803da: Generated new TSK 0
I20260926 16:41:30.589659 532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:30.594880 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 445
W20260926 16:41:30.600445 430 connection.cc:535] client connection to 127.0.0.1:40377 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40377): Connection reset by peer (error 104)
2026-09-26T16:41:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:30 dist-test-slave-p39x krb5kdc[544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:30 dist-test-slave-p39x krb5kdc[544](info): set up 2 sockets
Sep 26 16:41:30 dist-test-slave-p39x krb5kdc[544](info): commencing operation
krb5kdc: starting...
W20260926 16:41:31.645326 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:31 dist-test-slave-p39x krb5kdc[544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:31.772058 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:31.889458 557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:31.889806 557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:31.889889 557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:31.894150 557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:31.894250 557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:31.894296 557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:31.894356 557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:31.894394 557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:31.899842 557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:31.901005 557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:31.902983 557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:31.911975 563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:31.912051 557 server_base.cc:1034] running on GCE node
W20260926 16:41:31.911939 565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:31.911975 562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:31.912878 557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:31.914021 557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:31.915195 557 hybrid_clock.cc:648] HybridClock initialized: now 1790440891915165 us; error 43 us; skew 500 ppm
Sep 26 16:41:31 dist-test-slave-p39x krb5kdc[544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:31.927304 557 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:31.928931 557 webserver.cc:460] Webserver started at http://127.0.0.1:44257/ using document root <none> and password file <none>
I20260926 16:41:31.929702 557 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:31.929841 557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:31.930101 557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:31.932909 557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "8cc320846bca4f0ab9a0de222ffa7a58"
format_stamp: "Formatted at 2026-09-26 16:41:31 on dist-test-slave-p39x"
server_key: "dff8b8187ccb4ec522948ed9bb2198c2"
server_key_iv: "556d2d857508c1be428e6347ae43ce95"
server_key_version: "encryptionkey@0"
I20260926 16:41:31.933531 557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "8cc320846bca4f0ab9a0de222ffa7a58"
format_stamp: "Formatted at 2026-09-26 16:41:31 on dist-test-slave-p39x"
server_key: "dff8b8187ccb4ec522948ed9bb2198c2"
server_key_iv: "556d2d857508c1be428e6347ae43ce95"
server_key_version: "encryptionkey@0"
I20260926 16:41:31.937629 557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 16:41:31.940546 572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:31.941982 557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:31.942148 557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "8cc320846bca4f0ab9a0de222ffa7a58"
format_stamp: "Formatted at 2026-09-26 16:41:31 on dist-test-slave-p39x"
server_key: "dff8b8187ccb4ec522948ed9bb2198c2"
server_key_iv: "556d2d857508c1be428e6347ae43ce95"
server_key_version: "encryptionkey@0"
I20260926 16:41:31.942281 557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:31.947024 557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:31.951501 557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:31.951737 557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:31.959506 557 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33011
I20260926 16:41:31.959538 624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33011 every 8 connection(s)
I20260926 16:41:31.960772 557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:31.963553 625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:31.968986 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 557
I20260926 16:41:31.969136 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:31.969408 11898 external_mini_cluster.cc:1428] Setting key f5d2923256e164ef08bea4f3910bb2e8
I20260926 16:41:31.969810 625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58: Bootstrap starting.
I20260926 16:41:31.972595 625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:31.973431 625 log.cc:824] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:31.975322 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:31.970938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:39731 (local address 127.0.0.1:33011)
0926 16:41:31.971485 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:31.971500 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:31.971538 (+ 38us) server_negotiation.cc:407] Connection header received
0926 16:41:31.972241 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:31.972259 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:31.972670 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:31.973087 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:31.973925 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:31.973930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:31.973944 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:31.973955 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:31.974016 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:31.974036 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:31.974160 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:31.974265 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 16:41:31.974451 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":156,"threads_started":1}
I20260926 16:41:31.975860 625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58: No bootstrap required, opened a new log
I20260926 16:41:31.978665 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:31.980564 625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33011 } }
I20260926 16:41:31.980865 625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:31.980950 625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc320846bca4f0ab9a0de222ffa7a58, State: Initialized, Role: FOLLOWER
I20260926 16:41:31.981437 625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33011 } }
I20260926 16:41:31.981611 625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:31.981696 625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:31.981849 625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:31.982770 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:31.980983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:34313 (local address 127.0.0.1:33011)
0926 16:41:31.981111 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:31.981118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:31.981309 (+ 191us) server_negotiation.cc:407] Connection header received
0926 16:41:31.981483 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:31.981490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:31.981522 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:31.981603 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:31.982390 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:31.982394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:31.982396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:31.982407 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:31.982433 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:31.982452 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:31.982543 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:31.982612 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 16:41:31.982654 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:41:31.983165 625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33011 } }
I20260926 16:41:31.983564 625 leader_election.cc:302] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cc320846bca4f0ab9a0de222ffa7a58; no voters:
I20260926 16:41:31.984025 625 leader_election.cc:288] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:31.984104 635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:31.984326 635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [term 1 LEADER]: Becoming Leader. State: Replica: 8cc320846bca4f0ab9a0de222ffa7a58, State: Running, Role: LEADER
I20260926 16:41:31.984702 635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33011 } }
I20260926 16:41:31.985544 625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:31.986583 636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33011 } } }
I20260926 16:41:31.986776 636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:31.986984 637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cc320846bca4f0ab9a0de222ffa7a58. Latest consensus state: current_term: 1 leader_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cc320846bca4f0ab9a0de222ffa7a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33011 } } }
I20260926 16:41:31.987082 637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:31.987280 644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:31.990500 644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:31.996835 644 catalog_manager.cc:1347] Generated new cluster ID: 3635b8838bc141a28639f4b6ca3935d0
I20260926 16:41:31.996922 644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:32.009349 644 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:32.009717 644 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:32.015333 644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cc320846bca4f0ab9a0de222ffa7a58: Generated new TSK 0
I20260926 16:41:32.016114 644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:32.021255 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 557
2026-09-26T16:41:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:32 dist-test-slave-p39x krb5kdc[656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:32 dist-test-slave-p39x krb5kdc[656](info): set up 2 sockets
Sep 26 16:41:32 dist-test-slave-p39x krb5kdc[656](info): commencing operation
krb5kdc: starting...
W20260926 16:41:33.077299 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:33 dist-test-slave-p39x krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:33.208787 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:33.322626 669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:33.323064 669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:33.323170 669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:33.327764 669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:33.327848 669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:33.327868 669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:33.327886 669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:33.327901 669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:33.333657 669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:33.334782 669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:33.335805 669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:33.344767 669 server_base.cc:1034] running on GCE node
W20260926 16:41:33.344647 674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:33.344647 677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:33.344653 675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:33.345662 669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:33.346868 669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:33.348085 669 hybrid_clock.cc:648] HybridClock initialized: now 1790440893348072 us; error 32 us; skew 500 ppm
Sep 26 16:41:33 dist-test-slave-p39x krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440893, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:33.351018 669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:33.351995 669 webserver.cc:460] Webserver started at http://127.0.0.1:42143/ using document root <none> and password file <none>
I20260926 16:41:33.352666 669 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:33.352761 669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:33.353003 669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:33.355167 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "054f7790678b43138eb69991da3d0654"
format_stamp: "Formatted at 2026-09-26 16:41:33 on dist-test-slave-p39x"
server_key: "9ff4455180cd4366ee005ff5961fcccb"
server_key_iv: "958d118cb4ceac4b09f942c8c4e4463e"
server_key_version: "encryptionkey@0"
I20260926 16:41:33.355748 669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "054f7790678b43138eb69991da3d0654"
format_stamp: "Formatted at 2026-09-26 16:41:33 on dist-test-slave-p39x"
server_key: "9ff4455180cd4366ee005ff5961fcccb"
server_key_iv: "958d118cb4ceac4b09f942c8c4e4463e"
server_key_version: "encryptionkey@0"
I20260926 16:41:33.359555 669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:41:33.362010 684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:33.363236 669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:41:33.363389 669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "054f7790678b43138eb69991da3d0654"
format_stamp: "Formatted at 2026-09-26 16:41:33 on dist-test-slave-p39x"
server_key: "9ff4455180cd4366ee005ff5961fcccb"
server_key_iv: "958d118cb4ceac4b09f942c8c4e4463e"
server_key_version: "encryptionkey@0"
I20260926 16:41:33.363524 669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:33.380002 669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:33.383275 669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:33.383507 669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:33.390905 669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37777
I20260926 16:41:33.390946 736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37777 every 8 connection(s)
I20260926 16:41:33.391886 669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:33.394670 737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:33.395408 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 669
I20260926 16:41:33.395555 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:33.395849 11898 external_mini_cluster.cc:1428] Setting key b5de6f7baae7694cc42a75dfbc35e6e1
I20260926 16:41:33.401546 737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654: Bootstrap starting.
I20260926 16:41:33.404312 737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:33.405136 737 log.cc:824] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:41:33 dist-test-slave-p39x krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:41:33.407593 737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654: No bootstrap required, opened a new log
I20260926 16:41:33.411964 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:33.397283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:60889 (local address 127.0.0.1:37777)
0926 16:41:33.397717 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:33.397738 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:33.397795 (+ 57us) server_negotiation.cc:407] Connection header received
0926 16:41:33.398630 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:33.398655 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:33.399120 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:33.399501 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:33.400201 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:33.401683 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:33.402741 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:33.403448 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:33.403617 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:33.406566 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:33.406573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:33.406590 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:33.406621 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:33.408628 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:33.409145 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:33.409151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:33.409158 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:33.409310 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:33.409665 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:33.409671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:33.409674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:33.410253 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:33.410470 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:33.410746 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 16:41:33.410975 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":106,"threads_started":1}
I20260926 16:41:33.412358 737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054f7790678b43138eb69991da3d0654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260926 16:41:33.412626 737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:33.412725 737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054f7790678b43138eb69991da3d0654, State: Initialized, Role: FOLLOWER
I20260926 16:41:33.413192 737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054f7790678b43138eb69991da3d0654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260926 16:41:33.413331 737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:33.413421 737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:33.413542 737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:33.414721 737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054f7790678b43138eb69991da3d0654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260926 16:41:33.415117 737 leader_election.cc:302] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 054f7790678b43138eb69991da3d0654; no voters:
I20260926 16:41:33.415223 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:33.415452 737 leader_election.cc:288] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:33.415582 742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:33.415808 742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [term 1 LEADER]: Becoming Leader. State: Replica: 054f7790678b43138eb69991da3d0654, State: Running, Role: LEADER
I20260926 16:41:33.416199 742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054f7790678b43138eb69991da3d0654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } }
I20260926 16:41:33.416800 737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:33.417528 744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 054f7790678b43138eb69991da3d0654. Latest consensus state: current_term: 1 leader_uuid: "054f7790678b43138eb69991da3d0654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054f7790678b43138eb69991da3d0654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } } }
I20260926 16:41:33.417744 744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:33.417928 743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "054f7790678b43138eb69991da3d0654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054f7790678b43138eb69991da3d0654" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37777 } } }
I20260926 16:41:33.418159 743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:33.418327 756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:33.421674 756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:33.426872 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:33.417790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:54043 (local address 127.0.0.1:37777)
0926 16:41:33.417903 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:33.417909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:33.418146 (+ 237us) server_negotiation.cc:407] Connection header received
0926 16:41:33.418362 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:33.418366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:33.418411 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:33.418460 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:33.419503 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:33.420261 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:33.421322 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:33.421636 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:33.421753 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:33.423426 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:33.423430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:33.423433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:33.423460 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:33.425021 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:33.425523 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:33.425528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:33.425530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:33.425613 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:33.425947 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:33.425950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:33.425952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:33.426131 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:33.426286 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:33.426608 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 16:41:33.426715 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:41:33.429668 756 catalog_manager.cc:1347] Generated new cluster ID: db2c38def51c4a97b7e28431944d255d
I20260926 16:41:33.429736 756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:33.448939 756 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:33.449956 756 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:33.456780 756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 054f7790678b43138eb69991da3d0654: Generated new TSK 0
I20260926 16:41:33.457486 756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:33.461531 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 669
W20260926 16:41:33.467114 654 connection.cc:535] client connection to 127.0.0.1:37777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37777): Connection reset by peer (error 104)
2026-09-26T16:41:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:33 dist-test-slave-p39x krb5kdc[768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:33 dist-test-slave-p39x krb5kdc[768](info): set up 2 sockets
Sep 26 16:41:33 dist-test-slave-p39x krb5kdc[768](info): commencing operation
krb5kdc: starting...
W20260926 16:41:34.529170 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:34 dist-test-slave-p39x krb5kdc[768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:34.651777 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:34.763885 781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:34.764226 781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:34.764309 781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:34.768388 781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:34.768491 781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:34.768529 781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:34.768589 781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:34.768631 781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:34.774142 781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:34.775298 781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:34.776402 781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:34.785094 781 server_base.cc:1034] running on GCE node
W20260926 16:41:34.785080 786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:34.785254 789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:34.785085 787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:34.785864 781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:34.786917 781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:34.788084 781 hybrid_clock.cc:648] HybridClock initialized: now 1790440894788055 us; error 41 us; skew 500 ppm
Sep 26 16:41:34 dist-test-slave-p39x krb5kdc[768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:34.790812 781 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:34.791742 781 webserver.cc:460] Webserver started at http://127.0.0.1:34361/ using document root <none> and password file <none>
I20260926 16:41:34.792309 781 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:34.792425 781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:34.792655 781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:34.794657 781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "eb62e5ca420046c8b8ad728aedb1d71e"
format_stamp: "Formatted at 2026-09-26 16:41:34 on dist-test-slave-p39x"
server_key: "df87bc0a87127cb337678798f438a230"
server_key_iv: "0e7a02ec5a81127cd4aef09a95cbdc97"
server_key_version: "encryptionkey@0"
I20260926 16:41:34.795213 781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "eb62e5ca420046c8b8ad728aedb1d71e"
format_stamp: "Formatted at 2026-09-26 16:41:34 on dist-test-slave-p39x"
server_key: "df87bc0a87127cb337678798f438a230"
server_key_iv: "0e7a02ec5a81127cd4aef09a95cbdc97"
server_key_version: "encryptionkey@0"
I20260926 16:41:34.798962 781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:41:34.801501 796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:34.802811 781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:41:34.802981 781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "eb62e5ca420046c8b8ad728aedb1d71e"
format_stamp: "Formatted at 2026-09-26 16:41:34 on dist-test-slave-p39x"
server_key: "df87bc0a87127cb337678798f438a230"
server_key_iv: "0e7a02ec5a81127cd4aef09a95cbdc97"
server_key_version: "encryptionkey@0"
I20260926 16:41:34.803105 781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:34.819239 781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:34.822892 781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:34.823123 781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:34.831137 781 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41101
I20260926 16:41:34.831171 848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41101 every 8 connection(s)
I20260926 16:41:34.832175 781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:34.835039 849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:34.838534 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 781
I20260926 16:41:34.838693 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:34.838985 11898 external_mini_cluster.cc:1428] Setting key f5ad9620ad3856991d4dadb2de12881a
I20260926 16:41:34.841277 849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e: Bootstrap starting.
I20260926 16:41:34.843492 849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:34.844249 849 log.cc:824] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:34.846875 849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e: No bootstrap required, opened a new log
I20260926 16:41:34.848601 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:34.840471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:59485 (local address 127.0.0.1:41101)
0926 16:41:34.840910 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:34.840937 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:34.840971 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:41:34.841650 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:34.841669 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:34.842030 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:34.842357 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:34.842770 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:34.843822 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:34.844908 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:34.845567 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:34.845741 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:34.846513 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:34.846520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:34.846536 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:34.847038 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:34.847106 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:34.847137 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:34.847283 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:34.847487 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 16:41:34.847627 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":140,"threads_started":1}
I20260926 16:41:34.850975 849 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260926 16:41:34.851258 849 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:34.851331 849 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb62e5ca420046c8b8ad728aedb1d71e, State: Initialized, Role: FOLLOWER
I20260926 16:41:34.851910 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:34.851872 849 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260926 16:41:34.852054 849 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:34.852118 849 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:34.852213 849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:34.853631 849 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260926 16:41:34.854063 849 leader_election.cc:302] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb62e5ca420046c8b8ad728aedb1d71e; no voters:
I20260926 16:41:34.854457 849 leader_election.cc:288] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:34.854759 858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:34.855042 858 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [term 1 LEADER]: Becoming Leader. State: Replica: eb62e5ca420046c8b8ad728aedb1d71e, State: Running, Role: LEADER
I20260926 16:41:34.855432 858 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260926 16:41:34.855914 849 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:34.857235 860 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } } }
I20260926 16:41:34.857380 860 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:34.857764 870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:34.857738 861 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb62e5ca420046c8b8ad728aedb1d71e. Latest consensus state: current_term: 1 leader_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb62e5ca420046c8b8ad728aedb1d71e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } } }
I20260926 16:41:34.857885 861 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:34.860451 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:34.854496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:48225 (local address 127.0.0.1:41101)
0926 16:41:34.854626 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:34.854631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:34.854920 (+ 289us) server_negotiation.cc:407] Connection header received
0926 16:41:34.855115 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:34.855118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:34.855173 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:34.855227 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:34.855689 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:34.856383 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:34.858582 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:34.858910 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:34.859025 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:34.859919 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:34.859923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:34.859925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:34.859940 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:34.859968 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:34.859981 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:34.860073 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:34.860244 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 16:41:34.860281 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:41:34.861451 870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:34.867211 870 catalog_manager.cc:1347] Generated new cluster ID: c63c545d1bc44ca4a06f82a870e36653
I20260926 16:41:34.867309 870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:34.894950 870 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:34.896286 870 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:34.905524 870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb62e5ca420046c8b8ad728aedb1d71e: Generated new TSK 0
I20260926 16:41:34.906332 870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:34.921376 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 781
W20260926 16:41:34.926890 766 connection.cc:535] client connection to 127.0.0.1:41101 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41101): Connection reset by peer (error 104)
2026-09-26T16:41:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:34 dist-test-slave-p39x krb5kdc[880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:34 dist-test-slave-p39x krb5kdc[880](info): set up 2 sockets
Sep 26 16:41:34 dist-test-slave-p39x krb5kdc[880](info): commencing operation
krb5kdc: starting...
W20260926 16:41:35.961210 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:36 dist-test-slave-p39x krb5kdc[880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:41:36.091682 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45547
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:36.201999 893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:36.202378 893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:36.202467 893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:36.206651 893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:36.206766 893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:36.206804 893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:36.206845 893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:36.206877 893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:36.212435 893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45547
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:36.213593 893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:36.214679 893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:36.223820 893 server_base.cc:1034] running on GCE node
W20260926 16:41:36.223744 899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:36.223758 901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:36.223738 898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:36.224710 893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:36.225955 893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:36.227155 893 hybrid_clock.cc:648] HybridClock initialized: now 1790440896227144 us; error 33 us; skew 500 ppm
Sep 26 16:41:36 dist-test-slave-p39x krb5kdc[880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440896, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:36.230033 893 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:41:36.231012 893 webserver.cc:460] Webserver started at http://127.0.0.1:40671/ using document root <none> and password file <none>
I20260926 16:41:36.231607 893 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:36.231704 893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:36.231945 893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:36.234102 893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "f2d1e274a31a462b9a8b69b2cfa4539a"
format_stamp: "Formatted at 2026-09-26 16:41:36 on dist-test-slave-p39x"
server_key: "29aabcfa53f6f547bd4f12984e7de922"
server_key_iv: "79718b17a2c6ec421f5ba8d593595013"
server_key_version: "encryptionkey@0"
I20260926 16:41:36.234680 893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "f2d1e274a31a462b9a8b69b2cfa4539a"
format_stamp: "Formatted at 2026-09-26 16:41:36 on dist-test-slave-p39x"
server_key: "29aabcfa53f6f547bd4f12984e7de922"
server_key_iv: "79718b17a2c6ec421f5ba8d593595013"
server_key_version: "encryptionkey@0"
I20260926 16:41:36.238720 893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260926 16:41:36.241432 908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:36.242842 893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:36.243022 893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "f2d1e274a31a462b9a8b69b2cfa4539a"
format_stamp: "Formatted at 2026-09-26 16:41:36 on dist-test-slave-p39x"
server_key: "29aabcfa53f6f547bd4f12984e7de922"
server_key_iv: "79718b17a2c6ec421f5ba8d593595013"
server_key_version: "encryptionkey@0"
I20260926 16:41:36.243145 893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:36.248410 893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:36.252264 893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:36.252642 893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:36.260805 893 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45547
I20260926 16:41:36.260833 960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45547 every 8 connection(s)
I20260926 16:41:36.261922 893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:36.265530 961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:36.268220 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 893
I20260926 16:41:36.268476 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:36.268795 11898 external_mini_cluster.cc:1428] Setting key 038096d079dcdf6d976538b26457c308
I20260926 16:41:36.272639 961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a: Bootstrap starting.
I20260926 16:41:36.275127 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:36.270342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:34153 (local address 127.0.0.1:45547)
0926 16:41:36.270842 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:36.270862 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:36.270907 (+ 45us) server_negotiation.cc:407] Connection header received
0926 16:41:36.271739 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:36.271765 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:36.272178 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:36.272592 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:36.273602 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:36.273608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:36.273623 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:36.273636 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:36.273705 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:36.273737 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:36.273907 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:36.273996 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 16:41:36.274172 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":144,"threads_started":1}
I20260926 16:41:36.275565 961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:36.276530 961 log.cc:824] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:36.278903 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:36.279033 961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a: No bootstrap required, opened a new log
I20260926 16:41:36.283161 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:36.281087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.180:51629 (local address 127.0.0.1:45547)
0926 16:41:36.281227 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:36.281234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:36.281461 (+ 227us) server_negotiation.cc:407] Connection header received
0926 16:41:36.281599 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:36.281602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:36.281642 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:36.281734 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:36.282585 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:36.282652 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:36.282656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:36.282674 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:36.282706 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:36.282723 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:36.282836 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:36.282993 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 16:41:36.283037 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:41:36.283425 961 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45547 } }
I20260926 16:41:36.283701 961 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:36.283780 961 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d1e274a31a462b9a8b69b2cfa4539a, State: Initialized, Role: FOLLOWER
I20260926 16:41:36.284286 961 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45547 } }
I20260926 16:41:36.284503 961 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:36.284595 961 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:36.284698 961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:36.285766 961 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45547 } }
I20260926 16:41:36.286206 961 leader_election.cc:302] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2d1e274a31a462b9a8b69b2cfa4539a; no voters:
I20260926 16:41:36.286544 961 leader_election.cc:288] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:36.286717 971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:36.287014 971 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [term 1 LEADER]: Becoming Leader. State: Replica: f2d1e274a31a462b9a8b69b2cfa4539a, State: Running, Role: LEADER
I20260926 16:41:36.287382 971 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45547 } }
I20260926 16:41:36.287987 961 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:36.289218 973 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d1e274a31a462b9a8b69b2cfa4539a. Latest consensus state: current_term: 1 leader_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45547 } } }
I20260926 16:41:36.289376 973 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:36.289778 972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2d1e274a31a462b9a8b69b2cfa4539a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45547 } } }
I20260926 16:41:36.289935 972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:36.290261 980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:36.294229 980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:36.300590 980 catalog_manager.cc:1347] Generated new cluster ID: 768957a9bd094ded8c9148469fada4d4
I20260926 16:41:36.300685 980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:36.327139 980 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:36.327565 980 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:36.336755 980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2d1e274a31a462b9a8b69b2cfa4539a: Generated new TSK 0
I20260926 16:41:36.337708 980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:36.344521 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 893
2026-09-26T16:41:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:36 dist-test-slave-p39x krb5kdc[992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:36 dist-test-slave-p39x krb5kdc[992](info): set up 2 sockets
Sep 26 16:41:36 dist-test-slave-p39x krb5kdc[992](info): commencing operation
krb5kdc: starting...
W20260926 16:41:37.397326 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:37 dist-test-slave-p39x krb5kdc[992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:37.532586 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:45951
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:45029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:37.646062 1005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:37.646405 1005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:37.646492 1005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:37.650538 1005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:37.650645 1005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:37.650688 1005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:37.650753 1005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:37.650785 1005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:37.656138 1005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:45029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:45951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:37.657332 1005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:37.658408 1005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:37.667450 1005 server_base.cc:1034] running on GCE node
W20260926 16:41:37.667387 1013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:37.667460 1010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:37.667393 1011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:37.668224 1005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:37.669443 1005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:37.670636 1005 hybrid_clock.cc:648] HybridClock initialized: now 1790440897670610 us; error 56 us; skew 500 ppm
Sep 26 16:41:37 dist-test-slave-p39x krb5kdc[992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440897, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:37.679378 1005 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:37.680553 1005 webserver.cc:460] Webserver started at http://127.11.158.190:40755/ using document root <none> and password file <none>
I20260926 16:41:37.681175 1005 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:37.681272 1005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:37.681520 1005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:37.684283 1005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "e402826e1a37419fb0e76a6cf32449bf"
format_stamp: "Formatted at 2026-09-26 16:41:37 on dist-test-slave-p39x"
server_key: "d5c901d5b5b27e86c84ee2287a164347"
server_key_iv: "764795eab48004db7801322d82e5b272"
server_key_version: "encryptionkey@0"
I20260926 16:41:37.685029 1005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "e402826e1a37419fb0e76a6cf32449bf"
format_stamp: "Formatted at 2026-09-26 16:41:37 on dist-test-slave-p39x"
server_key: "d5c901d5b5b27e86c84ee2287a164347"
server_key_iv: "764795eab48004db7801322d82e5b272"
server_key_version: "encryptionkey@0"
I20260926 16:41:37.689090 1005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:41:37.691898 1020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:37.693260 1005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:41:37.693405 1005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "e402826e1a37419fb0e76a6cf32449bf"
format_stamp: "Formatted at 2026-09-26 16:41:37 on dist-test-slave-p39x"
server_key: "d5c901d5b5b27e86c84ee2287a164347"
server_key_iv: "764795eab48004db7801322d82e5b272"
server_key_version: "encryptionkey@0"
I20260926 16:41:37.693535 1005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:37.707762 1005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:37.711138 1005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:37.711359 1005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:37.718662 1005 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:45951
I20260926 16:41:37.718684 1072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:45951 every 8 connection(s)
I20260926 16:41:37.719674 1005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:37.722438 1073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:37.728189 1073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf: Bootstrap starting.
I20260926 16:41:37.729228 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1005
I20260926 16:41:37.729387 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:37.729676 11898 external_mini_cluster.cc:1428] Setting key ffe32bff9f9854ace264c802503c696d
I20260926 16:41:37.731181 1073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:37.731956 1073 log.cc:824] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:37.734107 1073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf: No bootstrap required, opened a new log
I20260926 16:41:37.737774 1073 raft_consensus.cc:348] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e402826e1a37419fb0e76a6cf32449bf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45951 } }
I20260926 16:41:37.738052 1073 raft_consensus.cc:374] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:37.738131 1073 raft_consensus.cc:729] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e402826e1a37419fb0e76a6cf32449bf, State: Initialized, Role: FOLLOWER
I20260926 16:41:37.738622 1073 consensus_queue.cc:260] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e402826e1a37419fb0e76a6cf32449bf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45951 } }
I20260926 16:41:37.738775 1073 raft_consensus.cc:388] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:37.738857 1073 raft_consensus.cc:482] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:37.738957 1073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 0 FOLLOWER]: Advancing to term 1
Sep 26 16:41:37 dist-test-slave-p39x krb5kdc[992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:37.740001 1073 raft_consensus.cc:504] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e402826e1a37419fb0e76a6cf32449bf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45951 } }
I20260926 16:41:37.740429 1073 leader_election.cc:302] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e402826e1a37419fb0e76a6cf32449bf; no voters:
I20260926 16:41:37.740840 1073 leader_election.cc:288] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:37.740990 1078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:37.741226 1078 raft_consensus.cc:686] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [term 1 LEADER]: Becoming Leader. State: Replica: e402826e1a37419fb0e76a6cf32449bf, State: Running, Role: LEADER
I20260926 16:41:37.741644 1078 consensus_queue.cc:237] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e402826e1a37419fb0e76a6cf32449bf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45951 } }
I20260926 16:41:37.742268 1073 sys_catalog.cc:564] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:37.743443 1080 sys_catalog.cc:455] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e402826e1a37419fb0e76a6cf32449bf. Latest consensus state: current_term: 1 leader_uuid: "e402826e1a37419fb0e76a6cf32449bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e402826e1a37419fb0e76a6cf32449bf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45951 } } }
I20260926 16:41:37.743582 1080 sys_catalog.cc:458] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:37.743963 1079 sys_catalog.cc:455] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e402826e1a37419fb0e76a6cf32449bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e402826e1a37419fb0e76a6cf32449bf" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45951 } } }
I20260926 16:41:37.744099 1079 sys_catalog.cc:458] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:37.744422 1087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:37.746387 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:37.730818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54735 (local address 127.11.158.190:45951)
0926 16:41:37.731395 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:37.731413 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:37.731453 (+ 40us) server_negotiation.cc:407] Connection header received
0926 16:41:37.732332 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:37.732390 (+ 58us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:37.732872 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:37.733292 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:37.734177 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:37.735339 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:37.736314 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:37.736909 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:37.737104 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:37.740191 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:37.740197 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:37.740214 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:37.740247 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:37.742598 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:37.743345 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:37.743352 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:37.743359 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:37.743517 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:37.743975 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:37.743979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:37.743981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:37.744577 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:37.744783 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:37.745366 (+ 583us) server_negotiation.cc:299] Negotiation successful
0926 16:41:37.745617 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":149,"threads_started":1}
I20260926 16:41:37.748095 1087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:37.749675 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:37.755127 1087 catalog_manager.cc:1347] Generated new cluster ID: ca1dfe477c8645db824323c7db7dc083
I20260926 16:41:37.755204 1087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:37.760783 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:37.751911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52757 (local address 127.11.158.190:45951)
0926 16:41:37.752041 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:37.752046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:37.752289 (+ 243us) server_negotiation.cc:407] Connection header received
0926 16:41:37.752487 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:37.752490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:37.752546 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:37.752618 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:37.753495 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:37.754024 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:37.754936 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:37.755316 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:37.755434 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:37.757206 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:37.757210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:37.757213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:37.757237 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:37.758903 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:37.759447 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:37.759450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:37.759452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:37.759530 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:37.759865 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:37.759868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:37.759870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:37.760067 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:37.760208 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:37.760526 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 16:41:37.760620 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:41:37.765797 1087 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:37.766889 1087 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:37.776855 1087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e402826e1a37419fb0e76a6cf32449bf: Generated new TSK 0
I20260926 16:41:37.777774 1087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:37.790820 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1005
W20260926 16:41:37.795986 990 connection.cc:535] client connection to 127.11.158.190:45951 recv error: Network error: failed to read from TLS socket (remote: 127.11.158.190:45951): Connection reset by peer (error 104)
2026-09-26T16:41:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:37 dist-test-slave-p39x krb5kdc[1104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:37 dist-test-slave-p39x krb5kdc[1104](info): set up 2 sockets
Sep 26 16:41:37 dist-test-slave-p39x krb5kdc[1104](info): commencing operation
krb5kdc: starting...
W20260926 16:41:38.845175 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:38 dist-test-slave-p39x krb5kdc[1104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:38.977057 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:45821
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:42623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:45821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:39.091137 1117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:39.091452 1117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:39.091552 1117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:39.095644 1117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:39.095731 1117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:39.095769 1117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:39.095824 1117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:39.095865 1117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:39.101083 1117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:42623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:45821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:45821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:39.102196 1117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:39.103235 1117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:39.111603 1123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:39.111693 1117 server_base.cc:1034] running on GCE node
W20260926 16:41:39.111593 1122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:39.111593 1125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:39.112540 1117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:39.113740 1117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:39.114926 1117 hybrid_clock.cc:648] HybridClock initialized: now 1790440899114901 us; error 39 us; skew 500 ppm
Sep 26 16:41:39 dist-test-slave-p39x krb5kdc[1104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440899, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:39.117714 1117 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:39.118635 1117 webserver.cc:460] Webserver started at http://127.11.158.190:34517/ using document root <none> and password file <none>
I20260926 16:41:39.119205 1117 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:39.119330 1117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:39.119568 1117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:39.121767 1117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "6f958ecaaa474f5bb57afb438a9578cd"
format_stamp: "Formatted at 2026-09-26 16:41:39 on dist-test-slave-p39x"
server_key: "a6a5d739d2b43b6dd62da0bd4481819c"
server_key_iv: "79c9ffb9f5ce137be68c5d67701c9f01"
server_key_version: "encryptionkey@0"
I20260926 16:41:39.122373 1117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "6f958ecaaa474f5bb57afb438a9578cd"
format_stamp: "Formatted at 2026-09-26 16:41:39 on dist-test-slave-p39x"
server_key: "a6a5d739d2b43b6dd62da0bd4481819c"
server_key_iv: "79c9ffb9f5ce137be68c5d67701c9f01"
server_key_version: "encryptionkey@0"
I20260926 16:41:39.127369 1117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 16:41:39.130034 1132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:39.131544 1117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:41:39.131714 1117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "6f958ecaaa474f5bb57afb438a9578cd"
format_stamp: "Formatted at 2026-09-26 16:41:39 on dist-test-slave-p39x"
server_key: "a6a5d739d2b43b6dd62da0bd4481819c"
server_key_iv: "79c9ffb9f5ce137be68c5d67701c9f01"
server_key_version: "encryptionkey@0"
I20260926 16:41:39.131855 1117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:39.157960 1117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:39.161652 1117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:39.161892 1117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:39.169135 1117 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:45821
I20260926 16:41:39.169158 1184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:45821 every 8 connection(s)
I20260926 16:41:39.170089 1117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:39.173012 1185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:39.174741 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1117
I20260926 16:41:39.174925 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:39.175197 11898 external_mini_cluster.cc:1428] Setting key 8c8ffd13f89e1147fc078a976eababb6
I20260926 16:41:39.179219 1185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd: Bootstrap starting.
I20260926 16:41:39.181460 1185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:39.182327 1185 log.cc:824] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:39.183983 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:39.176566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34809 (local address 127.11.158.190:45821)
0926 16:41:39.177038 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:39.177052 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:39.177086 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:41:39.177716 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:39.177733 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:39.178068 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:39.178410 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:39.178802 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:39.179811 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:39.180757 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:39.181222 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:39.181398 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:39.182301 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:39.182306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:39.182318 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:39.182747 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:39.182800 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:39.182823 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:39.182971 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:39.183149 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 16:41:39.183288 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":153,"threads_started":1}
I20260926 16:41:39.184425 1185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd: No bootstrap required, opened a new log
I20260926 16:41:39.186966 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:39.188611 1185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f958ecaaa474f5bb57afb438a9578cd" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45821 } }
I20260926 16:41:39.188913 1185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:39.189054 1185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f958ecaaa474f5bb57afb438a9578cd, State: Initialized, Role: FOLLOWER
I20260926 16:41:39.189553 1185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f958ecaaa474f5bb57afb438a9578cd" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45821 } }
I20260926 16:41:39.189716 1185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:39.189805 1185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:39.189911 1185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:39.191260 1185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f958ecaaa474f5bb57afb438a9578cd" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45821 } }
I20260926 16:41:39.191645 1185 leader_election.cc:302] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f958ecaaa474f5bb57afb438a9578cd; no voters:
I20260926 16:41:39.191994 1185 leader_election.cc:288] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:39.192075 1195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:39.192281 1195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [term 1 LEADER]: Becoming Leader. State: Replica: 6f958ecaaa474f5bb57afb438a9578cd, State: Running, Role: LEADER
I20260926 16:41:39.192673 1195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f958ecaaa474f5bb57afb438a9578cd" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45821 } }
I20260926 16:41:39.193463 1185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:39.193922 1195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f958ecaaa474f5bb57afb438a9578cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f958ecaaa474f5bb57afb438a9578cd" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45821 } } }
I20260926 16:41:39.194548 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:39.189130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35251 (local address 127.11.158.190:45821)
0926 16:41:39.189236 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:39.189239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:39.189475 (+ 236us) server_negotiation.cc:407] Connection header received
0926 16:41:39.189632 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:39.189635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:39.189688 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:39.189780 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:39.190104 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:39.190756 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:39.191940 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:39.192329 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:39.192469 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:39.193708 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:39.193712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:39.193715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:39.193728 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:39.193761 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:39.193776 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:39.193862 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:39.194326 (+ 464us) server_negotiation.cc:299] Negotiation successful
0926 16:41:39.194377 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 16:41:39.194712 1195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:39.194587 1196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f958ecaaa474f5bb57afb438a9578cd. Latest consensus state: current_term: 1 leader_uuid: "6f958ecaaa474f5bb57afb438a9578cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f958ecaaa474f5bb57afb438a9578cd" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 45821 } } }
I20260926 16:41:39.195061 1196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd [sys.catalog]: This master's current role is: LEADER
W20260926 16:41:39.198247 1210 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:39.198323 1210 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:39.198415 1204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:39.199158 1204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:39.206614 1204 catalog_manager.cc:1347] Generated new cluster ID: 9f9a8303b6384986ab8e5b5fc9a47d21
I20260926 16:41:39.206712 1204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:39.229585 1204 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:39.230947 1204 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:39.246834 1204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f958ecaaa474f5bb57afb438a9578cd: Generated new TSK 0
I20260926 16:41:39.247756 1204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:39.262794 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1117
W20260926 16:41:39.268429 1102 connection.cc:535] client connection to 127.11.158.190:45821 recv error: Network error: failed to read from TLS socket (remote: 127.11.158.190:45821): Connection reset by peer (error 104)
2026-09-26T16:41:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:39 dist-test-slave-p39x krb5kdc[1216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:39 dist-test-slave-p39x krb5kdc[1216](info): set up 2 sockets
Sep 26 16:41:39 dist-test-slave-p39x krb5kdc[1216](info): commencing operation
krb5kdc: starting...
W20260926 16:41:40.305507 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:40 dist-test-slave-p39x krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:40.431066 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:36403
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:36403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:40.542349 1229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:40.542636 1229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:40.542678 1229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:40.546856 1229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:40.546934 1229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:40.546955 1229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:40.546972 1229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:40.546988 1229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:40.552841 1229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:36403
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:36403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:40.554055 1229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:40.555125 1229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:40.564291 1235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:40.564495 1229 server_base.cc:1034] running on GCE node
W20260926 16:41:40.564277 1234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:40.564293 1237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:40.565317 1229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:40.566617 1229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:40.567821 1229 hybrid_clock.cc:648] HybridClock initialized: now 1790440900567800 us; error 37 us; skew 500 ppm
Sep 26 16:41:40 dist-test-slave-p39x krb5kdc[1216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440900, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:40.570904 1229 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:40.571907 1229 webserver.cc:460] Webserver started at http://127.11.158.190:37027/ using document root <none> and password file <none>
I20260926 16:41:40.572562 1229 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:40.572652 1229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:40.572892 1229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:40.574962 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "c5fcec3e142343c086fb4c5f770f2b41"
format_stamp: "Formatted at 2026-09-26 16:41:40 on dist-test-slave-p39x"
server_key: "5087beea59c3d6bc8d7bdef304234a10"
server_key_iv: "d5247959740c14f54fa02a47af1b9a18"
server_key_version: "encryptionkey@0"
I20260926 16:41:40.575526 1229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "c5fcec3e142343c086fb4c5f770f2b41"
format_stamp: "Formatted at 2026-09-26 16:41:40 on dist-test-slave-p39x"
server_key: "5087beea59c3d6bc8d7bdef304234a10"
server_key_iv: "d5247959740c14f54fa02a47af1b9a18"
server_key_version: "encryptionkey@0"
I20260926 16:41:40.579357 1229 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 16:41:40.582135 1244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:40.583511 1229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:41:40.583667 1229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "c5fcec3e142343c086fb4c5f770f2b41"
format_stamp: "Formatted at 2026-09-26 16:41:40 on dist-test-slave-p39x"
server_key: "5087beea59c3d6bc8d7bdef304234a10"
server_key_iv: "d5247959740c14f54fa02a47af1b9a18"
server_key_version: "encryptionkey@0"
I20260926 16:41:40.583801 1229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:40.588876 1229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:40.591980 1229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:40.592209 1229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:40.600095 1229 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:36403
I20260926 16:41:40.600107 1296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:36403 every 8 connection(s)
I20260926 16:41:40.601238 1229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:40.605223 1297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:40.607610 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1229
I20260926 16:41:40.607748 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:40.608084 11898 external_mini_cluster.cc:1428] Setting key 7aad94c073e9fc96a751f4d92e09603a
I20260926 16:41:40.611662 1297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41: Bootstrap starting.
I20260926 16:41:40.613557 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:40.609703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33827 (local address 127.11.158.190:36403)
0926 16:41:40.610151 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:40.610167 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:40.610204 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:41:40.610860 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:40.610877 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:40.611167 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:40.611458 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:40.612391 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:40.612396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:40.612408 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:40.612419 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:40.612477 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:40.612498 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:40.612609 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:40.612727 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 16:41:40.612869 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":149,"threads_started":1}
I20260926 16:41:40.614354 1297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:40.615263 1297 log.cc:824] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:40.616683 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:40.617473 1297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41: No bootstrap required, opened a new log
I20260926 16:41:40.620529 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:40.618774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50519 (local address 127.11.158.190:36403)
0926 16:41:40.618916 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:40.618923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:40.619119 (+ 196us) server_negotiation.cc:407] Connection header received
0926 16:41:40.619265 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:40.619268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:40.619308 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:40.619391 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:40.620123 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:40.620127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:40.620129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:40.620140 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:40.620171 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:40.620182 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:40.620272 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:40.620368 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 16:41:40.620409 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:41:40.621712 1297 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5fcec3e142343c086fb4c5f770f2b41" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36403 } }
I20260926 16:41:40.621941 1297 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:40.622015 1297 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5fcec3e142343c086fb4c5f770f2b41, State: Initialized, Role: FOLLOWER
I20260926 16:41:40.622442 1297 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5fcec3e142343c086fb4c5f770f2b41" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36403 } }
I20260926 16:41:40.622565 1297 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:40.622632 1297 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:40.622731 1297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:40.623643 1297 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5fcec3e142343c086fb4c5f770f2b41" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36403 } }
I20260926 16:41:40.623989 1297 leader_election.cc:302] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5fcec3e142343c086fb4c5f770f2b41; no voters:
I20260926 16:41:40.624289 1297 leader_election.cc:288] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:40.624538 1307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:40.624802 1307 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [term 1 LEADER]: Becoming Leader. State: Replica: c5fcec3e142343c086fb4c5f770f2b41, State: Running, Role: LEADER
I20260926 16:41:40.625125 1307 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5fcec3e142343c086fb4c5f770f2b41" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36403 } }
I20260926 16:41:40.625602 1297 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:40.626883 1308 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5fcec3e142343c086fb4c5f770f2b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5fcec3e142343c086fb4c5f770f2b41" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36403 } } }
I20260926 16:41:40.627027 1308 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:40.627705 1309 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5fcec3e142343c086fb4c5f770f2b41. Latest consensus state: current_term: 1 leader_uuid: "c5fcec3e142343c086fb4c5f770f2b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5fcec3e142343c086fb4c5f770f2b41" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 36403 } } }
I20260926 16:41:40.627902 1309 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41 [sys.catalog]: This master's current role is: LEADER
W20260926 16:41:40.630304 1322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:40.630409 1322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:40.630486 1318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:40.631408 1318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:40.637244 1318 catalog_manager.cc:1347] Generated new cluster ID: 77628a903311422bb20b0c2a0ec31c23
I20260926 16:41:40.637315 1318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:40.644354 1318 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:40.644681 1318 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:40.654621 1318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5fcec3e142343c086fb4c5f770f2b41: Generated new TSK 0
I20260926 16:41:40.655581 1318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:40.667694 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1229
2026-09-26T16:41:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:40 dist-test-slave-p39x krb5kdc[1328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:40 dist-test-slave-p39x krb5kdc[1328](info): set up 2 sockets
Sep 26 16:41:40 dist-test-slave-p39x krb5kdc[1328](info): commencing operation
krb5kdc: starting...
W20260926 16:41:41.745342 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:41 dist-test-slave-p39x krb5kdc[1328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:41.873281 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39789
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:39789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:41.985318 1341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:41.985656 1341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:41.985736 1341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:41.990583 1341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:41.990723 1341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:41.990800 1341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:41.990847 1341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:41.990916 1341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:41.996803 1341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:39789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:39789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:41.998000 1341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:41.999070 1341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:42.007788 1341 server_base.cc:1034] running on GCE node
W20260926 16:41:42.007830 1346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:42.007943 1347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:42.007830 1349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:42.008627 1341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:42.009786 1341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:42.011008 1341 hybrid_clock.cc:648] HybridClock initialized: now 1790440902010971 us; error 47 us; skew 500 ppm
Sep 26 16:41:42 dist-test-slave-p39x krb5kdc[1328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440902, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:42.014168 1341 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:42.015223 1341 webserver.cc:460] Webserver started at http://127.11.158.190:40911/ using document root <none> and password file <none>
I20260926 16:41:42.015825 1341 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:42.015924 1341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:42.016162 1341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:42.018309 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "73c194a27caf4155ac23fcab1a3590b1"
format_stamp: "Formatted at 2026-09-26 16:41:42 on dist-test-slave-p39x"
server_key: "975ffb4e723ef30103b8b223c9008869"
server_key_iv: "1943ff15e1053b1255892c50f66978ab"
server_key_version: "encryptionkey@0"
I20260926 16:41:42.018908 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "73c194a27caf4155ac23fcab1a3590b1"
format_stamp: "Formatted at 2026-09-26 16:41:42 on dist-test-slave-p39x"
server_key: "975ffb4e723ef30103b8b223c9008869"
server_key_iv: "1943ff15e1053b1255892c50f66978ab"
server_key_version: "encryptionkey@0"
I20260926 16:41:42.022706 1341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260926 16:41:42.025321 1356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:42.026551 1341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:41:42.026693 1341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "73c194a27caf4155ac23fcab1a3590b1"
format_stamp: "Formatted at 2026-09-26 16:41:42 on dist-test-slave-p39x"
server_key: "975ffb4e723ef30103b8b223c9008869"
server_key_iv: "1943ff15e1053b1255892c50f66978ab"
server_key_version: "encryptionkey@0"
I20260926 16:41:42.026808 1341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:42.070668 1341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:42.073943 1341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:42.074165 1341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:42.081900 1341 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:39789
I20260926 16:41:42.081930 1408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:39789 every 8 connection(s)
I20260926 16:41:42.083113 1341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:42.087260 1409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:42.090574 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1341
I20260926 16:41:42.090714 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:42.091020 11898 external_mini_cluster.cc:1428] Setting key bd75d1645814d92b29929809e32aa243
I20260926 16:41:42.095026 1409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1: Bootstrap starting.
I20260926 16:41:42.097388 1409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:42.098218 1409 log.cc:824] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:42.100603 1409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1: No bootstrap required, opened a new log
Sep 26 16:41:42 dist-test-slave-p39x krb5kdc[1328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:42.104732 1409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c194a27caf4155ac23fcab1a3590b1" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39789 } }
I20260926 16:41:42.105037 1409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:42.105103 1409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c194a27caf4155ac23fcab1a3590b1, State: Initialized, Role: FOLLOWER
I20260926 16:41:42.105597 1409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c194a27caf4155ac23fcab1a3590b1" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39789 } }
I20260926 16:41:42.105729 1409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:42.105816 1409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:42.105945 1409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:42.106308 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:42.092575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54431 (local address 127.11.158.190:39789)
0926 16:41:42.093115 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:42.093136 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:42.093177 (+ 41us) server_negotiation.cc:407] Connection header received
0926 16:41:42.094269 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:42.094308 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:42.094717 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:42.095064 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:42.095882 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:42.096983 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:42.097934 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:42.098453 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:42.098616 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:42.101381 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:42.101387 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:42.101404 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:42.101432 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:42.103204 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:42.103700 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:42.103705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:42.103709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:42.103816 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:42.104098 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:42.104101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:42.104102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:42.104559 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:42.104715 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:42.105065 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 16:41:42.105397 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":157,"threads_started":1}
I20260926 16:41:42.106959 1409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c194a27caf4155ac23fcab1a3590b1" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39789 } }
I20260926 16:41:42.107306 1409 leader_election.cc:302] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73c194a27caf4155ac23fcab1a3590b1; no voters:
I20260926 16:41:42.107620 1409 leader_election.cc:288] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:42.107733 1414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:42.107944 1414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [term 1 LEADER]: Becoming Leader. State: Replica: 73c194a27caf4155ac23fcab1a3590b1, State: Running, Role: LEADER
I20260926 16:41:42.108266 1414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c194a27caf4155ac23fcab1a3590b1" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39789 } }
I20260926 16:41:42.109045 1409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:42.109598 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:42.110240 1415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73c194a27caf4155ac23fcab1a3590b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c194a27caf4155ac23fcab1a3590b1" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39789 } } }
I20260926 16:41:42.110358 1415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:42.110711 1423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:42.111378 1416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73c194a27caf4155ac23fcab1a3590b1. Latest consensus state: current_term: 1 leader_uuid: "73c194a27caf4155ac23fcab1a3590b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c194a27caf4155ac23fcab1a3590b1" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 39789 } } }
I20260926 16:41:42.111521 1416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:42.114223 1423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:42.121117 1423 catalog_manager.cc:1347] Generated new cluster ID: fd1c08fffb4e43f9a64a375bfdc8d8c2
I20260926 16:41:42.121176 1423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:42.122053 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:42.112286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34769 (local address 127.11.158.190:39789)
0926 16:41:42.112432 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:42.112437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:42.113265 (+ 828us) server_negotiation.cc:407] Connection header received
0926 16:41:42.113556 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:42.113560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:42.113614 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:42.113705 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:42.114559 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:42.115360 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:42.116525 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:42.116808 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:42.116867 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:42.118379 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:42.118390 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:42.118393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:42.118419 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:42.120101 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:42.120647 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:42.120651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:42.120652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:42.120744 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:42.121070 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:42.121074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:42.121076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:42.121320 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:42.121466 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:42.121753 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 16:41:42.121883 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 16:41:42.133894 1423 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:42.134847 1423 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:42.140534 1423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73c194a27caf4155ac23fcab1a3590b1: Generated new TSK 0
I20260926 16:41:42.141285 1423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:42.146637 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1341
2026-09-26T16:41:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:42 dist-test-slave-p39x krb5kdc[1440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:42 dist-test-slave-p39x krb5kdc[1440](info): set up 2 sockets
Sep 26 16:41:42 dist-test-slave-p39x krb5kdc[1440](info): commencing operation
krb5kdc: starting...
W20260926 16:41:43.189282 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:43 dist-test-slave-p39x krb5kdc[1440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:43.320511 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33181
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:33181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:43.434347 1453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:43.434726 1453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:43.434818 1453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:43.439170 1453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:43.439283 1453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:43.439332 1453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:43.439393 1453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:43.439435 1453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:43.445335 1453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:33181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:33181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:43.446516 1453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:43.447597 1453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:43.456213 1458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:43.456207 1459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:43.456213 1461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:43.456488 1453 server_base.cc:1034] running on GCE node
I20260926 16:41:43.457299 1453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:43.458503 1453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:43.459704 1453 hybrid_clock.cc:648] HybridClock initialized: now 1790440903459691 us; error 32 us; skew 500 ppm
Sep 26 16:41:43 dist-test-slave-p39x krb5kdc[1440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440903, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:43.470171 1453 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:43.471225 1453 webserver.cc:460] Webserver started at http://127.11.158.190:38091/ using document root <none> and password file <none>
I20260926 16:41:43.471803 1453 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:43.471870 1453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:43.472064 1453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:43.474274 1453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad"
format_stamp: "Formatted at 2026-09-26 16:41:43 on dist-test-slave-p39x"
server_key: "018f9a4a042a16d2fd060243b2341d95"
server_key_iv: "02431383df9a511ea57b843868aa6116"
server_key_version: "encryptionkey@0"
I20260926 16:41:43.474834 1453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad"
format_stamp: "Formatted at 2026-09-26 16:41:43 on dist-test-slave-p39x"
server_key: "018f9a4a042a16d2fd060243b2341d95"
server_key_iv: "02431383df9a511ea57b843868aa6116"
server_key_version: "encryptionkey@0"
I20260926 16:41:43.478855 1453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260926 16:41:43.481659 1468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:43.483057 1453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:41:43.483197 1453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad"
format_stamp: "Formatted at 2026-09-26 16:41:43 on dist-test-slave-p39x"
server_key: "018f9a4a042a16d2fd060243b2341d95"
server_key_iv: "02431383df9a511ea57b843868aa6116"
server_key_version: "encryptionkey@0"
I20260926 16:41:43.483306 1453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:43.511685 1453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:43.514813 1453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:43.514976 1453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:43.522315 1453 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:33181
I20260926 16:41:43.522337 1520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:33181 every 8 connection(s)
I20260926 16:41:43.523279 1453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:43.526081 1521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:43.527910 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1453
I20260926 16:41:43.528059 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:43.528414 11898 external_mini_cluster.cc:1428] Setting key 2ba5b0602e003cf8d72c2869981e37bf
I20260926 16:41:43.533077 1521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad: Bootstrap starting.
I20260926 16:41:43.535619 1521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:43.536623 1521 log.cc:824] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:43.538529 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:43.529950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53569 (local address 127.11.158.190:33181)
0926 16:41:43.530442 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:43.530458 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:43.530496 (+ 38us) server_negotiation.cc:407] Connection header received
0926 16:41:43.531183 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:43.531201 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:43.531561 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:43.531932 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:43.532417 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:43.533490 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:43.534425 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:43.534917 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:43.535100 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:43.536127 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:43.536133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:43.536148 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:43.536687 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:43.536805 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:43.536856 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:43.536985 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:43.537177 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 16:41:43.537442 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":146,"threads_started":1}
I20260926 16:41:43.539237 1521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad: No bootstrap required, opened a new log
I20260926 16:41:43.541857 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:43.544209 1521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33181 } }
I20260926 16:41:43.544649 1521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:43.544755 1521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28fd9a75ebf0473ea2bb3e4c9f3a50ad, State: Initialized, Role: FOLLOWER
I20260926 16:41:43.545315 1521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33181 } }
I20260926 16:41:43.545495 1521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:43.545603 1521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:43.545723 1521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:43.546886 1521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33181 } }
I20260926 16:41:43.547313 1521 leader_election.cc:302] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28fd9a75ebf0473ea2bb3e4c9f3a50ad; no voters:
I20260926 16:41:43.547752 1521 leader_election.cc:288] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:43.547865 1531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:43.548072 1531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [term 1 LEADER]: Becoming Leader. State: Replica: 28fd9a75ebf0473ea2bb3e4c9f3a50ad, State: Running, Role: LEADER
I20260926 16:41:43.548475 1531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33181 } }
I20260926 16:41:43.548861 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:43.544061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52683 (local address 127.11.158.190:33181)
0926 16:41:43.544194 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:43.544199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:43.544391 (+ 192us) server_negotiation.cc:407] Connection header received
0926 16:41:43.544567 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:43.544569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:43.544632 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:43.544723 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:43.545190 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:43.545998 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:43.546927 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:43.547261 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:43.547384 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:43.548395 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:43.548399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:43.548401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:43.548415 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:43.548452 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:43.548463 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:43.548549 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:43.548676 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 16:41:43.548744 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:41:43.549345 1521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:43.549839 1531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28fd9a75ebf0473ea2bb3e4c9f3a50ad. Latest consensus state: current_term: 1 leader_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33181 } } }
I20260926 16:41:43.549981 1531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:43.550438 1535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:43.550822 1532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28fd9a75ebf0473ea2bb3e4c9f3a50ad" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 33181 } } }
I20260926 16:41:43.550932 1532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:43.554157 1535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:43.560702 1535 catalog_manager.cc:1347] Generated new cluster ID: ddfc7312b84141ffb51d5af37a4e7449
I20260926 16:41:43.560798 1535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:43.577669 1535 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:43.578754 1535 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:43.584264 1535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28fd9a75ebf0473ea2bb3e4c9f3a50ad: Generated new TSK 0
I20260926 16:41:43.585114 1535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:43.594347 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1453
2026-09-26T16:41:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:43 dist-test-slave-p39x krb5kdc[1552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:43 dist-test-slave-p39x krb5kdc[1552](info): set up 2 sockets
Sep 26 16:41:43 dist-test-slave-p39x krb5kdc[1552](info): commencing operation
krb5kdc: starting...
W20260926 16:41:44.657440 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:44 dist-test-slave-p39x krb5kdc[1552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:44.788537 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40413
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:36391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:40413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:44.902069 1565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:44.902400 1565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:44.902446 1565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:44.907001 1565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:44.907080 1565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:44.907101 1565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:44.907119 1565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:44.907135 1565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:44.912739 1565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:36391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:40413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:40413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:44.913772 1565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:44.914740 1565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:44.923271 1573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:44.923271 1571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:44.923430 1565 server_base.cc:1034] running on GCE node
W20260926 16:41:44.923611 1570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:44.924193 1565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:44.925335 1565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:44.926503 1565 hybrid_clock.cc:648] HybridClock initialized: now 1790440904926473 us; error 45 us; skew 500 ppm
Sep 26 16:41:44 dist-test-slave-p39x krb5kdc[1552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440904, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:44.929298 1565 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:44.930243 1565 webserver.cc:460] Webserver started at http://127.11.158.190:39479/ using document root <none> and password file <none>
I20260926 16:41:44.930801 1565 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:44.930898 1565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:44.931135 1565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:44.933276 1565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "89dea7ab56fe44d59d7df6f5c158d5e6"
format_stamp: "Formatted at 2026-09-26 16:41:44 on dist-test-slave-p39x"
server_key: "4bd6f568dc20129e493f9c7aa033a55f"
server_key_iv: "e1a8deca06dfed591b2f84d3bbc2fb2f"
server_key_version: "encryptionkey@0"
I20260926 16:41:44.933842 1565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "89dea7ab56fe44d59d7df6f5c158d5e6"
format_stamp: "Formatted at 2026-09-26 16:41:44 on dist-test-slave-p39x"
server_key: "4bd6f568dc20129e493f9c7aa033a55f"
server_key_iv: "e1a8deca06dfed591b2f84d3bbc2fb2f"
server_key_version: "encryptionkey@0"
I20260926 16:41:44.937575 1565 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:41:44.939932 1580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:44.941226 1565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:41:44.941395 1565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "89dea7ab56fe44d59d7df6f5c158d5e6"
format_stamp: "Formatted at 2026-09-26 16:41:44 on dist-test-slave-p39x"
server_key: "4bd6f568dc20129e493f9c7aa033a55f"
server_key_iv: "e1a8deca06dfed591b2f84d3bbc2fb2f"
server_key_version: "encryptionkey@0"
I20260926 16:41:44.941517 1565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:44.946172 1565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:44.949548 1565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:44.949757 1565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:44.956945 1565 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:40413
I20260926 16:41:44.956966 1632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:40413 every 8 connection(s)
I20260926 16:41:44.957890 1565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:44.960655 1633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:44.964994 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1565
I20260926 16:41:44.965142 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:44.965435 11898 external_mini_cluster.cc:1428] Setting key 61fcdf42f60a38b46315b6508a198f75
I20260926 16:41:44.966578 1633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6: Bootstrap starting.
I20260926 16:41:44.968954 1633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:44.969914 1633 log.cc:824] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:44.970964 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:44.966937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35975 (local address 127.11.158.190:40413)
0926 16:41:44.967405 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:44.967421 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:44.967457 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:41:44.968117 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:44.968134 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:44.968462 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:44.968824 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:44.969583 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:44.969588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:44.969599 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:44.969608 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:44.969658 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:44.969676 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:44.969759 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:44.969917 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 16:41:44.970088 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":127,"threads_started":1}
I20260926 16:41:44.971949 1633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6: No bootstrap required, opened a new log
I20260926 16:41:44.973865 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:44.976068 1633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40413 } }
I20260926 16:41:44.976415 1633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:41:44.976503 1633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89dea7ab56fe44d59d7df6f5c158d5e6, State: Initialized, Role: FOLLOWER
I20260926 16:41:44.977036 1633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40413 } }
I20260926 16:41:44.977218 1633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:41:44.977303 1633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:41:44.977437 1633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:44.977807 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:44.975945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58571 (local address 127.11.158.190:40413)
0926 16:41:44.976062 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:44.976066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:44.976273 (+ 207us) server_negotiation.cc:407] Connection header received
0926 16:41:44.976417 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:44.976422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:44.976471 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:44.976599 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:44.977329 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:44.977333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:44.977336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:41:44.977349 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:44.977381 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:41:44.977396 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:44.977485 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:44.977608 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 16:41:44.977653 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:41:44.978566 1633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40413 } }
I20260926 16:41:44.978953 1633 leader_election.cc:302] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89dea7ab56fe44d59d7df6f5c158d5e6; no voters:
I20260926 16:41:44.979221 1633 leader_election.cc:288] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:41:44.979388 1643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:44.979620 1643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [term 1 LEADER]: Becoming Leader. State: Replica: 89dea7ab56fe44d59d7df6f5c158d5e6, State: Running, Role: LEADER
I20260926 16:41:44.979964 1643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40413 } }
I20260926 16:41:44.980580 1633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:44.981618 1644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40413 } } }
I20260926 16:41:44.981752 1644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:44.981784 1645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89dea7ab56fe44d59d7df6f5c158d5e6. Latest consensus state: current_term: 1 leader_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89dea7ab56fe44d59d7df6f5c158d5e6" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 40413 } } }
I20260926 16:41:44.981905 1645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:44.982177 1652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:44.985560 1652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:44.991523 1652 catalog_manager.cc:1347] Generated new cluster ID: 759e55ac04f643feba765d168afc9acb
I20260926 16:41:44.991613 1652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:44.999300 1652 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:44.999643 1652 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:45.013833 1652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89dea7ab56fe44d59d7df6f5c158d5e6: Generated new TSK 0
I20260926 16:41:45.014781 1652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:41:45.027539 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1565
2026-09-26T16:41:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1431 ms)
[----------] 18 tests from AuthTokenIssuingTest (25997 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:45 dist-test-slave-p39x krb5kdc[1664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:45 dist-test-slave-p39x krb5kdc[1664](info): set up 2 sockets
Sep 26 16:41:45 dist-test-slave-p39x krb5kdc[1664](info): commencing operation
krb5kdc: starting...
W20260926 16:41:46.106328 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:46.236238 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38139
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38139,127.11.158.189:39597,127.11.158.188:33479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:46.350183 1677 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:41:46.350538 1677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:46.350615 1677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:46.350682 1677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:46.354790 1677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:46.354864 1677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:46.354884 1677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:46.354902 1677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:46.354918 1677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:46.360929 1677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38139,127.11.158.189:39597,127.11.158.188:33479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:38139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:46.362093 1677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:46.363199 1677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:46.372094 1677 server_base.cc:1034] running on GCE node
W20260926 16:41:46.372037 1682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:46.372042 1683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:46.372047 1685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:46.372925 1677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:46.374100 1677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:46.375321 1677 hybrid_clock.cc:648] HybridClock initialized: now 1790440906375290 us; error 46 us; skew 500 ppm
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:46.378302 1677 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:46.379231 1677 webserver.cc:460] Webserver started at http://127.11.158.190:36287/ using document root <none> and password file <none>
I20260926 16:41:46.379814 1677 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:46.379925 1677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:46.380162 1677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:46.382402 1677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "ca0fc31919bc4221bb617f8fc1a3151a"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "c449ee9b2be7e6bb9cc6469bbc0b87dc"
server_key_iv: "7ea6f6b3c992cd6ac88ee13a85587ec6"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.382992 1677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "ca0fc31919bc4221bb617f8fc1a3151a"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "c449ee9b2be7e6bb9cc6469bbc0b87dc"
server_key_iv: "7ea6f6b3c992cd6ac88ee13a85587ec6"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.387053 1677 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:41:46.389756 1692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:46.391094 1677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:41:46.391259 1677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "ca0fc31919bc4221bb617f8fc1a3151a"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "c449ee9b2be7e6bb9cc6469bbc0b87dc"
server_key_iv: "7ea6f6b3c992cd6ac88ee13a85587ec6"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.391407 1677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:46.412426 1677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:46.415998 1677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:46.416221 1677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:46.423414 1677 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:38139
I20260926 16:41:46.423743 1744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:38139 every 8 connection(s)
I20260926 16:41:46.424477 1677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:46.427546 1745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:46.431489 1745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:46.433001 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1677
I20260926 16:41:46.433156 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:46.433480 11898 external_mini_cluster.cc:1428] Setting key ee63c4b101cdcc91b6ec6cb19621adf6
WARNING: no policy specified for kudu/127.11.158.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.189@KRBTEST.COM" created.
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:46.450232 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.433964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38139 (local address 127.0.0.1:51882)
0926 16:41:46.434566 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.434604 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.435635 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.436406 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.436411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.436861 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.437100 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.437113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.438242 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.438247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.438971 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.438980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.439621 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.439625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.439713 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.440353 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.440370 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.444652 (+ 4282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.447023 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.447033 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.447043 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.447270 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.447597 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.447600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.447602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.447710 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.448451 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.448458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.448604 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.448786 (+ 182us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.449181 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":103,"spinlock_wait_cycles":12032,"thread_start_us":160,"threads_started":1}
I20260926 16:41:46.450232 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.433964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.11.158.190:38139)
0926 16:41:46.434412 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.434431 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.434717 (+ 286us) server_negotiation.cc:407] Connection header received
0926 16:41:46.435989 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.435994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.436173 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.436402 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.437252 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.438116 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.439091 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.439469 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.439634 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.444826 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.444832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.444843 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.444871 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.446912 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.447384 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.447390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.447392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.447497 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.447896 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.447902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.447905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.448289 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.448485 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.448849 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.449130 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":299,"spinlock_wait_cycles":7552,"thread_start_us":143,"threads_started":1}
I20260926 16:41:46.453027 1745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:46.453799 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.453483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40204)
0926 16:41:46.453670 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.453737 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 16:41:46.454001 1693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
I20260926 16:41:46.454774 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.454571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40220)
0926 16:41:46.454674 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.454719 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 16:41:46.454963 1745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:39597: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab.
Entry for principal kudu/127.11.158.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab.
I20260926 16:41:46.459754 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:39597
--webserver_interface=127.11.158.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab
--principal=kudu/127.11.158.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38139,127.11.158.189:39597,127.11.158.188:33479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:41:46.504299 1745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } attempt: 1
I20260926 16:41:46.505322 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.504979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40236)
0926 16:41:46.505151 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.505228 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 16:41:46.506136 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.505899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40242)
0926 16:41:46.506018 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.506072 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:41:46.506294 1745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:39597: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
W20260926 16:41:46.571581 1752 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:41:46.571826 1752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:46.571848 1752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:46.571939 1752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 16:41:46.574465 1745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } attempt: 2
I20260926 16:41:46.575279 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.574962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40254)
0926 16:41:46.575121 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.575191 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 16:41:46.576057 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.575855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40260)
0926 16:41:46.575956 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.576003 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 16:41:46.576222 1745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:39597: Network error: Client connection negotiation failed: client connection to 127.11.158.189:39597: connect: Connection refused (error 111)
W20260926 16:41:46.576557 1752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:46.576625 1752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:46.576668 1752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:46.576704 1752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:46.576737 1752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:46.581983 1752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38139,127.11.158.189:39597,127.11.158.188:33479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab
--principal=kudu/127.11.158.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:39597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.158.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:46.583010 1752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:46.583999 1752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:46.592226 1757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:46.592226 1760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:46.592308 1758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:46.592425 1752 server_base.cc:1034] running on GCE node
I20260926 16:41:46.593030 1752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:46.594060 1752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:46.595245 1752 hybrid_clock.cc:648] HybridClock initialized: now 1790440906595209 us; error 48 us; skew 500 ppm
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:46.597980 1752 init.cc:377] Logged in from keytab as kudu/127.11.158.189@KRBTEST.COM (short username kudu)
I20260926 16:41:46.598891 1752 webserver.cc:460] Webserver started at http://127.11.158.189:34275/ using document root <none> and password file <none>
I20260926 16:41:46.599440 1752 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:46.599529 1752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:46.599757 1752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:46.601994 1752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data/instance:
uuid: "cab5d8991f3541ed8494910dacefa4e5"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "5873b0e9281e1ee077f541d6c9118ac6"
server_key_iv: "3dfda160332ff6fdbdee7db8406a84a4"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.602595 1752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/wal/instance:
uuid: "cab5d8991f3541ed8494910dacefa4e5"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "5873b0e9281e1ee077f541d6c9118ac6"
server_key_iv: "3dfda160332ff6fdbdee7db8406a84a4"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.606457 1752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:41:46.608922 1767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:46.610073 1752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:46.610219 1752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/wal
uuid: "cab5d8991f3541ed8494910dacefa4e5"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "5873b0e9281e1ee077f541d6c9118ac6"
server_key_iv: "3dfda160332ff6fdbdee7db8406a84a4"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.610334 1752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:46.633368 1752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:46.636405 1752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:46.636603 1752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:46.643707 1752 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.189:39597
I20260926 16:41:46.643994 1819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.189:39597 every 8 connection(s)
I20260926 16:41:46.644738 1752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
I20260926 16:41:46.646067 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1752
I20260926 16:41:46.646167 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-1/wal/instance
I20260926 16:41:46.646477 11898 external_mini_cluster.cc:1428] Setting key 72599ac3023434ca5ddf6bfce33ba0ec
I20260926 16:41:46.647737 1820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:46.651458 1820 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.158.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.188@KRBTEST.COM" created.
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:46.665876 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.653418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.11.158.190:38139)
0926 16:41:46.653560 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.653564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.654563 (+ 999us) server_negotiation.cc:407] Connection header received
0926 16:41:46.655573 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.655576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.655636 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.655748 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.656999 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.657528 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.659192 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.659442 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.659536 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.662035 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.662038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.662041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.662068 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.663603 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.664238 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.664242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.664244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.664388 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.664741 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.664745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.664747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.664940 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.665051 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.665572 (+ 521us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.665699 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:41:46.666659 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.653876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38139 (local address 127.0.0.1:51896)
0926 16:41:46.654412 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.654453 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.655360 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.655924 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.655931 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.656254 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.656849 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.656865 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.657670 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.657678 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.659045 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.659058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.659586 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.659589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.659759 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.660443 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.660459 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.661850 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.663832 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.663838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.663854 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.664130 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.664489 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.664494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.664496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.664641 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.665050 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.665058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.665316 (+ 258us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.665503 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.665839 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":144,"threads_started":1}
I20260926 16:41:46.667445 1745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } attempt: 3
I20260926 16:41:46.668507 1820 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab.
Entry for principal kudu/127.11.158.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab.
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:41:46.674886 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:33479
--webserver_interface=127.11.158.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab
--principal=kudu/127.11.158.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:38139,127.11.158.189:39597,127.11.158.188:33479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:41:46.679193 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.669019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40274)
0926 16:41:46.669104 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.669118 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.669193 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.669577 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.669580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.669743 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.669857 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.669864 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.670874 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.670877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.671475 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.671485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.672247 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.672251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.672364 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.672887 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.672901 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.674244 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.677389 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.677395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.677397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.677610 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.677951 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.677957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.677961 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.678053 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.678636 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.678639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.678730 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.678856 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.679003 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 16:41:46.679313 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.667938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40266)
0926 16:41:46.668118 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.668134 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.668216 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.668773 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.668776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.669033 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.669156 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.669163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.670331 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.670334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.671588 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.671598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.672329 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.672352 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.672479 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.673066 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.673080 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.674851 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.677654 (+ 2803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.677661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.677665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.677876 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.678175 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.678178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.678181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.678281 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.678768 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.678772 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.678884 (+ 112us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.678997 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.679125 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 16:41:46.679406 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.669074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.11.158.189:39597)
0926 16:41:46.669373 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.669382 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.669393 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:41:46.669433 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.669435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.669467 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.669534 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.669973 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.670697 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.671590 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.672130 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.672243 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.674435 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.674446 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.674460 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.674485 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.677272 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.677710 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.677718 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.677721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.677839 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.678175 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.678178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.678180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.678517 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.678662 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.678949 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.679109 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":229,"thread_start_us":94,"threads_started":1}
I20260926 16:41:46.680207 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.668053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.11.158.189:39597)
0926 16:41:46.668381 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.668393 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.668415 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:41:46.668455 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.668460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.668642 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.668778 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.669273 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.670203 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.671702 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.672132 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.672290 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.675032 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.675039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.675044 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.675075 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.677529 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.678004 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.678010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.678011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.678077 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.678395 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.678400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.678401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.678632 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.678769 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.679990 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.680083 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":221,"thread_start_us":125,"threads_started":1}
I20260926 16:41:46.681387 1745 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:46.681624 1820 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:46.682186 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.681924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51054)
0926 16:41:46.682051 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.682108 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 16:41:46.682333 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.682067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51064)
0926 16:41:46.682187 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.682271 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 16:41:46.682535 1771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
I20260926 16:41:46.683275 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.682926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51066)
0926 16:41:46.683129 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.683193 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260926 16:41:46.683429 1745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33479: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
I20260926 16:41:46.683648 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.683411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51070)
0926 16:41:46.683550 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.683603 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260926 16:41:46.683832 1820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33479: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
I20260926 16:41:46.714601 1745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } attempt: 1
I20260926 16:41:46.715560 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.715179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51082)
0926 16:41:46.715372 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.715465 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 16:41:46.716429 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.716151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51084)
0926 16:41:46.716280 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.716363 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:41:46.716601 1745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33479: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
I20260926 16:41:46.733071 1820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } attempt: 1
I20260926 16:41:46.733956 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.733665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51086)
0926 16:41:46.733813 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.733876 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 16:41:46.734700 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.734488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51088)
0926 16:41:46.734595 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.734646 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 16:41:46.734843 1820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33479: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
W20260926 16:41:46.790431 1827 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:41:46.790804 1827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:46.790871 1827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:46.790953 1827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 16:41:46.791778 1745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } attempt: 2
I20260926 16:41:46.792644 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.792319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51104)
0926 16:41:46.792499 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.792570 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260926 16:41:46.793383 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.793164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51120)
0926 16:41:46.793274 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.793322 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 16:41:46.793509 1745 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33479: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
W20260926 16:41:46.795560 1827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:46.795645 1827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:46.795692 1827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:46.795786 1827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:46.795830 1827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:46.801735 1827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:38139,127.11.158.189:39597,127.11.158.188:33479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab
--principal=kudu/127.11.158.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:33479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.158.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:46.803063 1827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:46.803025 1820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } attempt: 2
I20260926 16:41:46.803880 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.803598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51128)
0926 16:41:46.803744 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.803805 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 16:41:46.804433 1827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:46.804641 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.804429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51142)
0926 16:41:46.804529 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.804585 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 16:41:46.804765 1820 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33479: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33479: connect: Connection refused (error 111)
W20260926 16:41:46.812777 1836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:46.812800 1834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:46.812880 1827 server_base.cc:1034] running on GCE node
W20260926 16:41:46.812777 1833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:46.813660 1827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:46.814790 1827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:46.815972 1827 hybrid_clock.cc:648] HybridClock initialized: now 1790440906815955 us; error 34 us; skew 500 ppm
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:46.818797 1827 init.cc:377] Logged in from keytab as kudu/127.11.158.188@KRBTEST.COM (short username kudu)
I20260926 16:41:46.819752 1827 webserver.cc:460] Webserver started at http://127.11.158.188:42419/ using document root <none> and password file <none>
I20260926 16:41:46.820328 1827 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:46.820451 1827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:46.820679 1827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:46.822729 1827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data/instance:
uuid: "c59b1ce69eb24d47a3f863362f502be1"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "fac83905b71203caa21e7ce44564b0d4"
server_key_iv: "847e3b36a0cfc7fb52651c48483d63c1"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.823315 1827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/wal/instance:
uuid: "c59b1ce69eb24d47a3f863362f502be1"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "fac83905b71203caa21e7ce44564b0d4"
server_key_iv: "847e3b36a0cfc7fb52651c48483d63c1"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.827128 1827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:41:46.829667 1843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:46.830971 1827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:41:46.831123 1827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/wal
uuid: "c59b1ce69eb24d47a3f863362f502be1"
format_stamp: "Formatted at 2026-09-26 16:41:46 on dist-test-slave-p39x"
server_key: "fac83905b71203caa21e7ce44564b0d4"
server_key_iv: "847e3b36a0cfc7fb52651c48483d63c1"
server_key_version: "encryptionkey@0"
I20260926 16:41:46.831243 1827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:46.856521 1827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:46.859684 1827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:46.859907 1827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:46.867201 1827 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.188:33479
I20260926 16:41:46.867540 1895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.188:33479 every 8 connection(s)
I20260926 16:41:46.868206 1827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
I20260926 16:41:46.871083 1896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:46.873100 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 1827
I20260926 16:41:46.873219 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440622937842-11898-0/minicluster-data/master-2/wal/instance
I20260926 16:41:46.873504 11898 external_mini_cluster.cc:1428] Setting key d0e2132f9d3829e0883456ce6f4e9afe
I20260926 16:41:46.875653 1896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:41:46.888120 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.878009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.11.158.190:38139)
0926 16:41:46.878144 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.878148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.878395 (+ 247us) server_negotiation.cc:407] Connection header received
0926 16:41:46.878503 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.878507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.878558 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.878646 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.879331 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.879851 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.880954 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.881257 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.881384 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.883810 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.883817 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.883822 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.883865 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.885707 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.886271 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.886275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.886277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.886368 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.886733 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.886740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.886744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.887020 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.887125 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.887669 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.887798 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:41:46.888970 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.877909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:38139 (local address 127.0.0.1:51902)
0926 16:41:46.878253 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.878302 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.878421 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.878698 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.878704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.879033 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.879187 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.879197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.879972 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.879975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.880837 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.880847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.881411 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.881414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.881520 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.882010 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.882025 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.883557 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.885851 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.885860 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.885871 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.886164 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.886495 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.886503 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.886505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.886634 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.887131 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.887137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.887440 (+ 303us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.887614 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.887944 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":30,"thread_start_us":105,"threads_started":1}
I20260926 16:41:46.888970 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.875048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.11.158.188:33479)
0926 16:41:46.875539 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.875558 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.875595 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:41:46.876458 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.876478 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.876874 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.877220 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.878132 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.879321 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.880285 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.880882 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.881058 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.884582 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.884594 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.884610 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.884641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.886333 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.886779 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.886783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.886784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.886850 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.887121 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.887124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.887126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.887462 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.887686 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.887882 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.887997 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":342,"thread_start_us":147,"threads_started":1}
I20260926 16:41:46.890985 1896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:46.892663 1745 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } attempt: 3
I20260926 16:41:46.895917 1820 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } attempt: 3
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:41:46.902370 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.891752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.11.158.189:39597)
0926 16:41:46.891877 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.891880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.891892 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:41:46.891930 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.891932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.891969 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.892064 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.892523 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.893231 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.894263 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.894621 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.894744 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.898215 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.898221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.898226 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.898258 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.900139 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.900720 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.900723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.900725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.900810 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.901361 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.901365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.901367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.901575 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.901648 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.902020 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.902151 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:41:46.902956 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.891614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51912 (local address 127.11.158.190:38139)
0926 16:41:46.891730 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.891733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.891747 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:41:46.891909 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.891912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.891964 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.892068 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.892737 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.893435 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.894187 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.894505 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.894607 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.897917 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.897925 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.897940 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.897975 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.900068 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.900640 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.900646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.900651 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.900737 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.901250 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.901256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.901261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.901489 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.901717 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.902410 (+ 693us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.902807 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:41:46.903439 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.893030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51166)
0926 16:41:46.893185 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.893198 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.893260 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.893617 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.893620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.893793 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.893877 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.893883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.894817 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.894821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.895614 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.895628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.896606 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.896609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.896668 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.897084 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.897122 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.898509 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.901755 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.901758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.901761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.902050 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.902400 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.902403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.902405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.902496 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.902872 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.902876 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.902976 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.903088 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.903223 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260926 16:41:46.903570 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.893095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.11.158.188:33479)
0926 16:41:46.893412 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.893416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.893426 (+ 10us) server_negotiation.cc:407] Connection header received
0926 16:41:46.893454 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.893457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.893492 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.893981 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.894054 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.894698 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.895835 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.896215 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.896402 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.898643 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.898647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.898652 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.899034 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.901643 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.902188 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.902192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.902195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.902284 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.902606 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.902609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.902610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.902756 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.902878 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.903153 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.903403 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":246}
I20260926 16:41:46.904208 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.891500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:39597 (local address 127.0.0.1:40290)
0926 16:41:46.891657 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.891672 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.891747 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.892077 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.892079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.892258 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.892398 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.892406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.893347 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.893351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.894134 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.894140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.894873 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.894876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.895008 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.895488 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.895502 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.898025 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.900245 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.900248 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.900250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.900586 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.901129 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.901133 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.901135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.901232 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.901707 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.901710 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.901810 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.903140 (+ 1330us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.903317 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:41:46.904891 1896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:46.907574 1745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a: Bootstrap starting.
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
Sep 26 16:41:46 dist-test-slave-p39x krb5kdc[1664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:41:46.910658 1745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:46.911541 1745 log.cc:824] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:46.913682 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.905501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.11.158.188:33479)
0926 16:41:46.905600 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.905604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.905614 (+ 10us) server_negotiation.cc:407] Connection header received
0926 16:41:46.905639 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.905641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.905679 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.905763 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.906105 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.906576 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.907358 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.907572 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.907680 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.909931 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.909937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.909943 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.909976 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.911616 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.912149 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.912152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.912154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.912222 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.912584 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.912588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.912590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.912903 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.912983 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.913374 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.913505 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 16:41:46.914206 1900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.905373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51176)
0926 16:41:46.905452 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.905464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.905532 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.905774 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.905776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.905939 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.905998 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.906003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.906679 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.906685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.907188 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.907194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.907678 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.907681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.907763 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.908256 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.908270 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.909782 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.911779 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.911782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.911784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.912043 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.912385 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.912390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.912393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.912481 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.913053 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.913056 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.913160 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.913876 (+ 716us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.914017 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":2}
I20260926 16:41:46.914254 1745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a: No bootstrap required, opened a new log
I20260926 16:41:46.915589 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.904662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40292 (local address 127.11.158.189:39597)
0926 16:41:46.904785 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.904789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.904943 (+ 154us) server_negotiation.cc:407] Connection header received
0926 16:41:46.905086 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.905089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.905136 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.905213 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.906033 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.906769 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.907882 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.908183 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.908256 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.911127 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.911132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.911138 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.911173 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.912955 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.913506 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.913509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.913511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.913594 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.914210 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.914214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.914216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.914764 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.914944 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.915331 (+ 387us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.915451 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:41:46.916118 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.896290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33479 (local address 127.0.0.1:51170)
0926 16:41:46.896430 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:46.896444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:46.896513 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:46.898139 (+ 1626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:46.898142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:46.898328 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:46.898431 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.898439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.899900 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.899904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.900984 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:46.900997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.901730 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.901737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:46.901834 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.902338 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:46.902356 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:46.904669 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:46.914375 (+ 9706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.914378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.914380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.914757 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.915117 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:46.915120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:46.915122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:46.915212 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:46.915671 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:46.915674 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:46.915759 (+ 85us) client_negotiation.cc:769] Sending connection context
0926 16:41:46.915835 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 16:41:46.915970 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 16:41:46.916662 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.897501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.11.158.188:33479)
0926 16:41:46.897873 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.897884 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.897910 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:41:46.897945 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.897949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.897995 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.898547 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.898646 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.899775 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.901141 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.901580 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.902420 (+ 840us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.912038 (+ 9618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.912042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.912044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.912068 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.914240 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.914917 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.914923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.914926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.915010 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.915360 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.915366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.915371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.915561 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.916282 (+ 721us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.916383 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.916502 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":145,"threads_started":1}
I20260926 16:41:46.917459 1896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: Bootstrap starting.
I20260926 16:41:46.919196 1745 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
I20260926 16:41:46.919435 1745 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca0fc31919bc4221bb617f8fc1a3151a, State: Initialized, Role: FOLLOWER
I20260926 16:41:46.919764 1745 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
I20260926 16:41:46.920022 1896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:46.920394 1820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: Bootstrap starting.
I20260926 16:41:46.920323 1903 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.920480 1903 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:46.920872 1896 log.cc:824] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:46.921181 1745 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:46.922798 1896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: No bootstrap required, opened a new log
I20260926 16:41:46.922946 1820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:46.924791 1820 log.cc:824] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: Log is configured to *not* fsync() on all Append() calls
W20260926 16:41:46.926805 1915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:46.926892 1915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:46.927747 1896 raft_consensus.cc:348] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
I20260926 16:41:46.927930 1896 raft_consensus.cc:729] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59b1ce69eb24d47a3f863362f502be1, State: Initialized, Role: FOLLOWER
I20260926 16:41:46.928169 1896 consensus_queue.cc:260] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
I20260926 16:41:46.928587 1918 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.928705 1918 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:46.929342 1896 sys_catalog.cc:564] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:46.926252 1677 master_runner.cc:186] Error getting master registration for 127.11.158.189:39597: OK, instance_id { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" instance_seqno: 1790440906641809 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:41:46.930153 1677 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:41:46.931154 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:41:46.932672 1719 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ca0fc31919bc4221bb617f8fc1a3151a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51912
I20260926 16:41:46.932896 1719 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 16:41:46.933000 1719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:46.934441 1719 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
W20260926 16:41:46.934595 1929 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:46.934665 1929 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:46.934947 1820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: No bootstrap required, opened a new log
I20260926 16:41:46.935425 1719 leader_election.cc:288] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [CANDIDATE]: Term 1 election: Requested vote from peers cab5d8991f3541ed8494910dacefa4e5 (127.11.158.189:39597), c59b1ce69eb24d47a3f863362f502be1 (127.11.158.188:33479)
I20260926 16:41:46.936110 1870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c59b1ce69eb24d47a3f863362f502be1"
I20260926 16:41:46.936352 1870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:46.937704 1870 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca0fc31919bc4221bb617f8fc1a3151a in term 1.
I20260926 16:41:46.939705 1820 raft_consensus.cc:348] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
I20260926 16:41:46.940011 1820 raft_consensus.cc:729] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab5d8991f3541ed8494910dacefa4e5, State: Initialized, Role: FOLLOWER
I20260926 16:41:46.940416 1820 consensus_queue.cc:260] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
I20260926 16:41:46.940652 1696 leader_election.cc:302] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c59b1ce69eb24d47a3f863362f502be1, ca0fc31919bc4221bb617f8fc1a3151a; no voters:
I20260926 16:41:46.940984 1903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:46.941224 1903 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [term 1 LEADER]: Becoming Leader. State: Replica: ca0fc31919bc4221bb617f8fc1a3151a, State: Running, Role: LEADER
I20260926 16:41:46.941565 1903 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } }
I20260926 16:41:46.942080 1820 sys_catalog.cc:564] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:46.943384 1903 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca0fc31919bc4221bb617f8fc1a3151a. Latest consensus state: current_term: 1 leader_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.943498 1903 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:46.944631 1794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cab5d8991f3541ed8494910dacefa4e5"
I20260926 16:41:46.945534 1794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:46.946871 1794 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca0fc31919bc4221bb617f8fc1a3151a in term 1.
I20260926 16:41:46.948704 1943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:46.949673 1943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:46.951143 1827 master_runner.cc:263] No leader master found from master c59b1ce69eb24d47a3f863362f502be1
I20260926 16:41:46.951208 1827 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:41:46.951961 1935 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.952095 1935 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:46.960021 1870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [term 1 FOLLOWER]: Refusing update from remote peer ca0fc31919bc4221bb617f8fc1a3151a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:46.960698 1903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:41:46.961972 1949 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:46.962112 1949 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:46.964916 1794 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [term 1 FOLLOWER]: Refusing update from remote peer ca0fc31919bc4221bb617f8fc1a3151a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:46.965605 1903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:46.969798 1935 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca0fc31919bc4221bb617f8fc1a3151a. Latest consensus state: current_term: 1 leader_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.969934 1935 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:46.969919 1918 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca0fc31919bc4221bb617f8fc1a3151a. Latest consensus state: current_term: 1 leader_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.970041 1918 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:46.971920 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.953841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.11.158.189:39597)
0926 16:41:46.953945 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.953949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.962492 (+ 8543us) server_negotiation.cc:407] Connection header received
0926 16:41:46.962668 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.962672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.962718 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.962767 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.963738 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.964553 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.965856 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.966201 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.966273 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.968144 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.968147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.968150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.968176 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.969949 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.970577 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.970580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.970582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.970667 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.970961 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.970964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.970966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.971174 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.971271 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.971653 (+ 382us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.971769 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:41:46.977044 1936 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.977212 1936 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:46.977365 1950 mvcc.cc:205] Tried to move back new op lower bound from 7333645954900598784 to 7333645954836176896. Current Snapshot: MvccSnapshot[applied={T|T < 7333645954900598784}]
I20260926 16:41:46.978760 1918 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.978909 1918 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:46.979053 1955 mvcc.cc:205] Tried to move back new op lower bound from 7333645954900598784 to 7333645954836176896. Current Snapshot: MvccSnapshot[applied={T|T < 7333645954900598784}]
I20260926 16:41:46.979594 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.952200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.11.158.190:38139)
0926 16:41:46.952307 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.952312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.954176 (+ 1864us) server_negotiation.cc:407] Connection header received
0926 16:41:46.954351 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.954354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.954401 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.954449 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.956745 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.957539 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.972642 (+ 15103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.973006 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.973082 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.975095 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.975099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.975102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.975135 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.976858 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.977822 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.977826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.977829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.977919 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.978354 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.978358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.978360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.978582 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.978694 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.979311 (+ 617us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.979437 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:41:46.987826 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:46.952873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51184 (local address 127.11.158.188:33479)
0926 16:41:46.952977 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:46.952981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:46.955575 (+ 2594us) server_negotiation.cc:407] Connection header received
0926 16:41:46.955724 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:46.955727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:46.955791 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:46.955840 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:46.970537 (+ 14697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.971350 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.973580 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:46.973950 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:46.974023 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:46.984176 (+ 10153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:46.984181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:46.984184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:46.984214 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:46.985972 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.986477 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.986481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.986483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.986567 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:46.986860 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:46.986864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:46.986865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:46.987078 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:46.987191 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:46.987528 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 16:41:46.987652 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:41:46.990038 1903 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.990175 1903 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:46.991411 1935 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca0fc31919bc4221bb617f8fc1a3151a" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 38139 } } peers { permanent_uuid: "cab5d8991f3541ed8494910dacefa4e5" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 39597 } } peers { permanent_uuid: "c59b1ce69eb24d47a3f863362f502be1" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33479 } } }
I20260926 16:41:46.991536 1935 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:46.992290 1957 mvcc.cc:205] Tried to move back new op lower bound from 7333645954900598784 to 7333645954836176896. Current Snapshot: MvccSnapshot[applied={T|T < 7333645954900598784}]
I20260926 16:41:46.998584 1943 catalog_manager.cc:1347] Generated new cluster ID: 2efcc965133541459050305fd309b083
I20260926 16:41:46.998667 1943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:47.021350 1943 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:47.022964 1943 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:47.054996 1943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca0fc31919bc4221bb617f8fc1a3151a: Generated new TSK 0
I20260926 16:41:47.055979 1943 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:41:47.090799 1747 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.087859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51926 (local address 127.11.158.190:38139)
0926 16:41:47.087993 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.087998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.089325 (+ 1327us) server_negotiation.cc:407] Connection header received
0926 16:41:47.089748 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.089753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.089809 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.089866 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.090660 (+ 794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51926: BlockingRecv error: recv got EOF from 127.0.0.1:51926 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 16:41:47.091966 1901 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.088646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.11.158.188:33479)
0926 16:41:47.088758 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.088763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.089664 (+ 901us) server_negotiation.cc:407] Connection header received
0926 16:41:47.091161 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.091166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.091227 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.091296 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.091850 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51196: BlockingRecv error: recv got EOF from 127.0.0.1:51196 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 16:41:47.094177 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.092818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.11.158.189:39597)
0926 16:41:47.092928 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.092932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.093255 (+ 323us) server_negotiation.cc:407] Connection header received
0926 16:41:47.093456 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.093459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.093506 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.093555 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.094056 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40306: BlockingRecv error: recv got EOF from 127.0.0.1:40306 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260926 16:41:47.098986 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.097378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.11.158.189:39597)
0926 16:41:47.097480 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.097484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.097952 (+ 468us) server_negotiation.cc:407] Connection header received
0926 16:41:47.098178 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.098181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.098280 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.098352 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.098889 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40316: BlockingRecv error: recv got EOF from 127.0.0.1:40316 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 16:41:47.104622 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.102705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40318 (local address 127.11.158.189:39597)
0926 16:41:47.102825 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.102830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.103432 (+ 602us) server_negotiation.cc:407] Connection header received
0926 16:41:47.103658 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.103662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.103720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.103778 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.104501 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40318: BlockingRecv error: recv got EOF from 127.0.0.1:40318 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260926 16:41:47.110912 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.109176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.11.158.189:39597)
0926 16:41:47.109303 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.109308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.109896 (+ 588us) server_negotiation.cc:407] Connection header received
0926 16:41:47.110107 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.110110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.110166 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.110224 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.110795 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40322: BlockingRecv error: recv got EOF from 127.0.0.1:40322 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 16:41:47.119544 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.117703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.11.158.189:39597)
0926 16:41:47.117873 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.117878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.118455 (+ 577us) server_negotiation.cc:407] Connection header received
0926 16:41:47.118685 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.118689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.118750 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.118811 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.119424 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40326: BlockingRecv error: recv got EOF from 127.0.0.1:40326 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260926 16:41:47.132217 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.130375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.11.158.189:39597)
0926 16:41:47.130513 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.130518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.131148 (+ 630us) server_negotiation.cc:407] Connection header received
0926 16:41:47.131392 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.131395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.131453 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.131508 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.132092 (+ 584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40336: BlockingRecv error: recv got EOF from 127.0.0.1:40336 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 16:41:47.152205 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.150837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.11.158.189:39597)
0926 16:41:47.150972 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.150977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.151246 (+ 269us) server_negotiation.cc:407] Connection header received
0926 16:41:47.151428 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.151432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.151481 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.151572 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.152068 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40346: BlockingRecv error: recv got EOF from 127.0.0.1:40346 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 16:41:47.187757 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.186561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.11.158.189:39597)
0926 16:41:47.186753 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.186757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.186828 (+ 71us) server_negotiation.cc:407] Connection header received
0926 16:41:47.186971 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.186973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.187021 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.187094 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.187633 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40360: BlockingRecv error: recv got EOF from 127.0.0.1:40360 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260926 16:41:47.255496 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.254356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.11.158.189:39597)
0926 16:41:47.254528 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.254532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.254567 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:41:47.254718 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.254720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.254768 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.254848 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.255362 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40376: BlockingRecv error: recv got EOF from 127.0.0.1:40376 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260926 16:41:47.387166 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.385947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.11.158.189:39597)
0926 16:41:47.386126 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.386130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.386257 (+ 127us) server_negotiation.cc:407] Connection header received
0926 16:41:47.386420 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.386423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.386472 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.386560 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.387034 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40380: BlockingRecv error: recv got EOF from 127.0.0.1:40380 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260926 16:41:47.646747 1825 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:47.645627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.11.158.189:39597)
0926 16:41:47.645773 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:47.645778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:47.645913 (+ 135us) server_negotiation.cc:407] Connection header received
0926 16:41:47.646032 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:47.646035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:47.646089 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:47.646186 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:47.646641 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40392: BlockingRecv error: recv got EOF from 127.0.0.1:40392 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:41:47.936110 1929 catalog_manager.cc:1259] Loaded cluster ID: 2efcc965133541459050305fd309b083
I20260926 16:41:47.936190 1929 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: loading cluster ID for follower catalog manager: success
I20260926 16:41:47.937897 1929 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: acquiring CA information for follower catalog manager: success
I20260926 16:41:47.939009 1929 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c59b1ce69eb24d47a3f863362f502be1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:41:47.963488 1949 catalog_manager.cc:1259] Loaded cluster ID: 2efcc965133541459050305fd309b083
I20260926 16:41:47.963608 1949 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: loading cluster ID for follower catalog manager: success
I20260926 16:41:47.965353 1949 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: acquiring CA information for follower catalog manager: success
I20260926 16:41:47.966513 1949 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cab5d8991f3541ed8494910dacefa4e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:41:48.165097 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:48.160991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.11.158.189:39597)
0926 16:41:48.161252 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:48.161259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:48.161341 (+ 82us) server_negotiation.cc:407] Connection header received
0926 16:41:48.161530 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:48.161533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:48.161666 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:48.161789 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:48.162053 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.162586 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.163478 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.163712 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.163821 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:48.164169 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:48.164674 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:48.164809 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:48.164876 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 16:41:48.164932 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":72,"threads_started":1}
I20260926 16:41:48.172124 2032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:48.168442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.11.158.188:33479)
0926 16:41:48.168681 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:48.168685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:48.168698 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:41:48.168771 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:48.168773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:48.168886 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:48.168982 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:48.169212 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.169705 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.170620 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.170890 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.170971 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:48.171253 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:48.171662 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:48.171751 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:48.171860 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 16:41:48.171929 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
I20260926 16:41:48.180015 2032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:48.175792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.11.158.188:33479)
0926 16:41:48.175899 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:48.175904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:48.175957 (+ 53us) server_negotiation.cc:407] Connection header received
0926 16:41:48.176083 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:48.176086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:48.176136 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:48.176228 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:48.177501 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.178001 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.178884 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.179086 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.179200 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:48.179482 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:48.179559 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:48.179608 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:48.179781 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 16:41:48.179845 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:41:48.180167 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:48.175211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.11.158.189:39597)
0926 16:41:48.175343 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:48.175349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:48.175723 (+ 374us) server_negotiation.cc:407] Connection header received
0926 16:41:48.175938 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:48.175942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:48.175999 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:48.176051 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:48.176512 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.177346 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.178814 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.179147 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.179248 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:48.179632 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:48.179711 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:48.179812 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:48.179970 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 16:41:48.180023 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:41:48.180707 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:48.175296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.11.158.190:38139)
0926 16:41:48.175550 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:48.175575 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:48.175595 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:41:48.175696 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:48.175699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:48.175881 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:48.176371 (+ 490us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:48.176519 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.177342 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.178931 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:48.179282 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:48.179389 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:48.179718 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:48.180064 (+ 346us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:48.180474 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:48.180511 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 16:41:48.180566 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
I20260926 16:41:48.184125 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1677
I20260926 16:41:48.191099 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1752
I20260926 16:41:48.196856 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 1827
2026-09-26T16:41:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3170 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:41:48 dist-test-slave-p39x krb5kdc[2043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:41:48 dist-test-slave-p39x krb5kdc[2043](info): set up 2 sockets
Sep 26 16:41:48 dist-test-slave-p39x krb5kdc[2043](info): commencing operation
krb5kdc: starting...
W20260926 16:41:49.237413 11898 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/test-user.keytab.
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.158.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
Entry for principal kudu/127.11.158.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab.
I20260926 16:41:49.357828 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:41541
--webserver_interface=127.11.158.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:45509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:41541,127.11.158.189:42679,127.11.158.188:33577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:41:49.466044 2056 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:41:49.466346 2056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:49.466370 2056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:49.466409 2056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:49.470371 2056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:49.470427 2056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:49.470445 2056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:49.470463 2056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:49.470476 2056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:49.475812 2056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:45509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:41541,127.11.158.189:42679,127.11.158.188:33577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.190.keytab
--principal=kudu/127.11.158.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.190:41541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.158.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:49.476859 2056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:49.477834 2056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:49.486745 2056 server_base.cc:1034] running on GCE node
W20260926 16:41:49.486706 2064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:49.486706 2062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:49.486708 2061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:49.487660 2056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:49.488865 2056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:49.490048 2056 hybrid_clock.cc:648] HybridClock initialized: now 1790440909490017 us; error 46 us; skew 500 ppm
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:49.499133 2056 init.cc:377] Logged in from keytab as kudu/127.11.158.190@KRBTEST.COM (short username kudu)
I20260926 16:41:49.500197 2056 webserver.cc:460] Webserver started at http://127.11.158.190:37037/ using document root <none> and password file <none>
I20260926 16:41:49.500837 2056 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:49.500905 2056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:49.501137 2056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:49.503221 2056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data/instance:
uuid: "7da15438994a4d3880a0a0b778527428"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "12d422892ad1baf74b63e517587b1df4"
server_key_iv: "2af379ecaf27a3371e2dcb373fa0c885"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.503789 2056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/wal/instance:
uuid: "7da15438994a4d3880a0a0b778527428"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "12d422892ad1baf74b63e517587b1df4"
server_key_iv: "2af379ecaf27a3371e2dcb373fa0c885"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.507491 2056 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:41:49.510346 2071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:49.511705 2056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:41:49.511858 2056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/wal
uuid: "7da15438994a4d3880a0a0b778527428"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "12d422892ad1baf74b63e517587b1df4"
server_key_iv: "2af379ecaf27a3371e2dcb373fa0c885"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.511981 2056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:49.525982 2056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:49.529196 2056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:49.529405 2056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:49.536669 2056 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.190:41541
I20260926 16:41:49.536959 2124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.190:41541 every 8 connection(s)
I20260926 16:41:49.537755 2056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/data/info.pb
I20260926 16:41:49.540527 2125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:49.544111 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 2056
I20260926 16:41:49.544246 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-0/wal/instance
I20260926 16:41:49.544602 11898 external_mini_cluster.cc:1428] Setting key 38fe08a300fb90dd6149cf3d725137de
I20260926 16:41:49.544634 2125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.158.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.189@KRBTEST.COM" created.
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:49.562680 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.547431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:41541 (local address 127.0.0.1:46184)
0926 16:41:49.548028 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.548085 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:49.549420 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:49.550199 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:49.550208 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:49.550725 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:49.551128 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.551148 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.552361 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.552366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.553636 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.553646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.554313 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.554315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.554466 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.555003 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:49.555021 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:49.557387 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:49.559550 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.559559 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.559570 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.559801 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.560143 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.560148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.560150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.560264 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.560929 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:49.560936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:49.561083 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 16:41:49.561250 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 16:41:49.561648 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":117,"thread_start_us":149,"threads_started":1}
I20260926 16:41:49.562680 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.547431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46184 (local address 127.11.158.190:41541)
0926 16:41:49.548018 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:49.548036 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:49.548210 (+ 174us) server_negotiation.cc:407] Connection header received
0926 16:41:49.549768 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:49.549778 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:49.549981 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:49.550139 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:49.551310 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.552202 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.553761 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.554168 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.554301 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.557590 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:49.557600 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:49.557618 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:49.557649 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:49.559408 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.559936 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.559941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.559942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.560046 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.560385 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.560388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.560390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.560789 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:49.560959 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:49.561324 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 16:41:49.561594 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":464,"spinlock_wait_cycles":56576,"thread_start_us":153,"threads_started":1}
I20260926 16:41:49.565090 2125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:49.565845 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.565574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38030)
0926 16:41:49.565697 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.565774 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 16:41:49.566049 2072 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
I20260926 16:41:49.566995 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.566787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38036)
0926 16:41:49.566889 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.566938 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 16:41:49.567171 2125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:42679: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab.
Entry for principal kudu/127.11.158.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab.
I20260926 16:41:49.573590 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:42679
--webserver_interface=127.11.158.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab
--principal=kudu/127.11.158.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:45509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:41541,127.11.158.189:42679,127.11.158.188:33577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:41:49.616451 2125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } attempt: 1
I20260926 16:41:49.617426 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.617117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38040)
0926 16:41:49.617270 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.617344 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 16:41:49.618271 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.618024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38046)
0926 16:41:49.618164 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.618206 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 16:41:49.618436 2125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:42679: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
I20260926 16:41:49.686633 2125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } attempt: 2
I20260926 16:41:49.687575 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.687255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38060)
0926 16:41:49.687421 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.687487 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 16:41:49.688429 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.688152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38074)
0926 16:41:49.688254 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.688301 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 16:41:49.688624 2125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.189:42679: Network error: Client connection negotiation failed: client connection to 127.11.158.189:42679: connect: Connection refused (error 111)
W20260926 16:41:49.689572 2132 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:41:49.689864 2132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:49.689940 2132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:49.690012 2132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:49.694314 2132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:49.694386 2132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:49.694407 2132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:49.694425 2132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:49.694440 2132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:49.700134 2132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:45509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:41541,127.11.158.189:42679,127.11.158.188:33577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.189.keytab
--principal=kudu/127.11.158.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.189:42679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.158.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:49.701326 2132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:49.702378 2132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:41:49.711265 2140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:49.711279 2138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:49.711267 2137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:49.711539 2132 server_base.cc:1034] running on GCE node
I20260926 16:41:49.712186 2132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:49.713330 2132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:49.714521 2132 hybrid_clock.cc:648] HybridClock initialized: now 1790440909714504 us; error 34 us; skew 500 ppm
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:49.717625 2132 init.cc:377] Logged in from keytab as kudu/127.11.158.189@KRBTEST.COM (short username kudu)
I20260926 16:41:49.718542 2132 webserver.cc:460] Webserver started at http://127.11.158.189:40549/ using document root <none> and password file <none>
I20260926 16:41:49.719127 2132 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:49.719218 2132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:49.719453 2132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:49.721596 2132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data/instance:
uuid: "e8e87a42a6e349369c0d805c548e04b1"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "fdffadc73c997825a4a75723d1d2f85e"
server_key_iv: "82923f8c378aceb686a1cf2ab1f73508"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.722175 2132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/wal/instance:
uuid: "e8e87a42a6e349369c0d805c548e04b1"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "fdffadc73c997825a4a75723d1d2f85e"
server_key_iv: "82923f8c378aceb686a1cf2ab1f73508"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.725903 2132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:41:49.728260 2147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:49.729683 2132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:41:49.729863 2132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/wal
uuid: "e8e87a42a6e349369c0d805c548e04b1"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "fdffadc73c997825a4a75723d1d2f85e"
server_key_iv: "82923f8c378aceb686a1cf2ab1f73508"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.729988 2132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:49.754974 2132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:49.758308 2132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:49.758545 2132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:49.765960 2132 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.189:42679
I20260926 16:41:49.766251 2199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.189:42679 every 8 connection(s)
I20260926 16:41:49.767050 2132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/data/info.pb
I20260926 16:41:49.769989 2200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:49.770390 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 2132
I20260926 16:41:49.770531 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-1/wal/instance
I20260926 16:41:49.770855 11898 external_mini_cluster.cc:1428] Setting key d7d587ed16b3520f8e8d7d09fbf8d274
I20260926 16:41:49.774552 2200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.158.188@KRBTEST.COM; defaulting to no policy
I20260926 16:41:49.779806 2125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.158.188@KRBTEST.COM" created.
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:41:49.790741 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.776497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.11.158.190:41541)
0926 16:41:49.776662 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:49.776667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:49.777654 (+ 987us) server_negotiation.cc:407] Connection header received
0926 16:41:49.779019 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:49.779022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:49.779076 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:49.779152 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:49.780497 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.781194 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.782868 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.783207 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.783311 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.786188 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:49.786192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:49.786195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:49.786221 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:49.788600 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.789169 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.789173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.789174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.789255 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.789611 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.789614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.789616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.789803 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:49.789909 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:49.790439 (+ 530us) server_negotiation.cc:299] Negotiation successful
0926 16:41:49.790562 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:41:49.791167 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.780255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38086)
0926 16:41:49.780483 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.780497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:49.780605 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:49.781194 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:49.781198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:49.781418 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:49.781542 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.781548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.782550 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.782555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.783301 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.783308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.783910 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.784191 (+ 281us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.784278 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.784798 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:49.784817 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:49.786421 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:49.789396 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.789412 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.789414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.789673 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.790052 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.790057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.790061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.790155 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.790726 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:49.790732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:49.790843 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 16:41:49.790927 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 16:41:49.791054 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260926 16:41:49.791728 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.780391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38086 (local address 127.11.158.189:42679)
0926 16:41:49.780794 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:49.780803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:49.780823 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:41:49.780856 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:49.780860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:49.781006 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:49.781142 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:49.781657 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.782435 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.783421 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.783786 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.783914 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.786855 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:49.786865 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:49.786885 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:49.786918 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:49.789259 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.789822 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.789830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.789833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.789916 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.790274 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.790277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.790279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.790597 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:49.791386 (+ 789us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:49.791497 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 16:41:49.791609 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":177,"threads_started":1}
I20260926 16:41:49.791728 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.776922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:41541 (local address 127.0.0.1:46192)
0926 16:41:49.777464 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.777525 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:49.778773 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:49.779332 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:49.779339 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:49.779735 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:49.780331 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.780383 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.781324 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.781327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.782724 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.782739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.783384 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.783391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.783629 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.784271 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:49.784306 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:49.786017 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:49.788731 (+ 2714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.788740 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.788751 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.789061 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.789358 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.789362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.789364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.789506 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.789909 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:49.789918 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:49.790183 (+ 265us) client_negotiation.cc:769] Sending connection context
0926 16:41:49.790371 (+ 188us) client_negotiation.cc:240] Negotiation successful
0926 16:41:49.790693 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":126,"threads_started":1}
I20260926 16:41:49.792991 2200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:49.793865 2125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:49.794560 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.794314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45840)
0926 16:41:49.794440 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.794484 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 16:41:49.795243 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.795008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45854)
0926 16:41:49.795143 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.795190 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260926 16:41:49.795375 2125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33577: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.158.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab.
Entry for principal kudu/127.11.158.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab.
I20260926 16:41:49.799511 11898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskupRbDL/build/debug/bin/kudu
/tmp/dist-test-taskupRbDL/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:33577
--webserver_interface=127.11.158.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab
--principal=kudu/127.11.158.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.158.148:45509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.158.190:41541,127.11.158.189:42679,127.11.158.188:33577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:41:49.801484 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.793561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38088)
0926 16:41:49.793655 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.793857 (+ 202us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:49.793920 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:49.794163 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:49.794166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:49.794348 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:49.794459 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.794466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.795395 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.795408 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.796007 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:49.796016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.796531 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.796559 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:49.796667 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.797043 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:49.797056 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:49.798217 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:49.799928 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.799935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.799940 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.800201 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.800512 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:49.800518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:49.800523 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:49.800627 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:49.801118 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:49.801120 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:49.801193 (+ 73us) client_negotiation.cc:769] Sending connection context
0926 16:41:49.801274 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:41:49.801368 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 16:41:49.801801 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.793644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.11.158.189:42679)
0926 16:41:49.793752 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:49.793756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:49.793897 (+ 141us) server_negotiation.cc:407] Connection header received
0926 16:41:49.794014 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:49.794017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:49.794059 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:49.794137 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:49.794579 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.795276 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.796125 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:49.796414 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:49.796508 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:49.798334 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:49.798337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:49.798338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:49.798358 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:49.799818 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.800295 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.800298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.800299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.800420 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:49.800746 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:49.800748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:49.800750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:49.801010 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:49.801121 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:49.801355 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 16:41:49.801481 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:41:49.802527 2200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:49.803253 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.802945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45860)
0926 16:41:49.803082 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.803173 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 16:41:49.803877 2150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
I20260926 16:41:49.804831 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.804561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45870)
0926 16:41:49.804696 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.804757 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 16:41:49.805006 2200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33577: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
I20260926 16:41:49.826553 2125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } attempt: 1
I20260926 16:41:49.827445 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.827091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45876)
0926 16:41:49.827261 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.827353 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 16:41:49.828311 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.828073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45884)
0926 16:41:49.828184 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.828231 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:41:49.828609 2125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33577: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
I20260926 16:41:49.854315 2200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } attempt: 1
I20260926 16:41:49.855372 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.855011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45896)
0926 16:41:49.855200 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.855278 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 16:41:49.856174 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.855956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45900)
0926 16:41:49.856064 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.856112 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:41:49.856369 2200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33577: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
I20260926 16:41:49.903843 2125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } attempt: 2
I20260926 16:41:49.904747 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.904464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45912)
0926 16:41:49.904600 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.904670 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 16:41:49.905568 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.905333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45928)
0926 16:41:49.905460 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.905515 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 16:41:49.905776 2125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33577: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
W20260926 16:41:49.915519 2206 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:41:49.915998 2206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:41:49.916059 2206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:41:49.916141 2206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:41:49.920845 2206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:41:49.920943 2206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:41:49.920996 2206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:41:49.921056 2206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:41:49.921092 2206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:41:49.924532 2200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } attempt: 2
I20260926 16:41:49.925420 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.925099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45934)
0926 16:41:49.925259 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.925331 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 16:41:49.926162 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:49.925952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45936)
0926 16:41:49.926054 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:49.926103 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 16:41:49.926288 2200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.158.188:33577: Network error: Client connection negotiation failed: client connection to 127.11.158.188:33577: connect: Connection refused (error 111)
I20260926 16:41:49.928067 2206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.158.148:45509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.158.190:41541,127.11.158.189:42679,127.11.158.188:33577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/krb5kdc/kudu_127.11.158.188.keytab
--principal=kudu/127.11.158.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.158.188:33577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.158.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:41:49.929243 2206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:41:49.930295 2206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:41:49.939634 2206 server_base.cc:1034] running on GCE node
W20260926 16:41:49.939656 2215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:49.939611 2212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:41:49.939611 2213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:41:49.940562 2206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:41:49.941726 2206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:41:49.942948 2206 hybrid_clock.cc:648] HybridClock initialized: now 1790440909942914 us; error 48 us; skew 500 ppm
Sep 26 16:41:49 dist-test-slave-p39x krb5kdc[2043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:41:49.946075 2206 init.cc:377] Logged in from keytab as kudu/127.11.158.188@KRBTEST.COM (short username kudu)
I20260926 16:41:49.947075 2206 webserver.cc:460] Webserver started at http://127.11.158.188:34641/ using document root <none> and password file <none>
I20260926 16:41:49.947695 2206 fs_manager.cc:359] Metadata directory not provided
I20260926 16:41:49.947791 2206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:41:49.948024 2206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:41:49.950261 2206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data/instance:
uuid: "5a23d8bc174041d6b730691e4d13a4c5"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "64e4b502d0d897646b0b8433d18865dc"
server_key_iv: "c3ebb20d757ae9b9c8ae5ea2bdbac545"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.950871 2206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/wal/instance:
uuid: "5a23d8bc174041d6b730691e4d13a4c5"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "64e4b502d0d897646b0b8433d18865dc"
server_key_iv: "c3ebb20d757ae9b9c8ae5ea2bdbac545"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.954993 2206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:41:49.957605 2222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:41:49.959072 2206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:41:49.959237 2206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data,/tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/wal
uuid: "5a23d8bc174041d6b730691e4d13a4c5"
format_stamp: "Formatted at 2026-09-26 16:41:49 on dist-test-slave-p39x"
server_key: "64e4b502d0d897646b0b8433d18865dc"
server_key_iv: "c3ebb20d757ae9b9c8ae5ea2bdbac545"
server_key_version: "encryptionkey@0"
I20260926 16:41:49.959373 2206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:41:49.991365 2206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:41:49.995250 2206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:41:49.995492 2206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:41:50.003059 2206 rpc_server.cc:307] RPC server started. Bound to: 127.11.158.188:33577
I20260926 16:41:50.003356 2274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.158.188:33577 every 8 connection(s)
I20260926 16:41:50.004108 2206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/data/info.pb
I20260926 16:41:50.004967 2125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } attempt: 3
I20260926 16:41:50.007380 11898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskupRbDL/build/debug/bin/kudu as pid 2206
I20260926 16:41:50.007483 2275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:41:50.007532 11898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskupRbDL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440622937842-11898-0/minicluster-data/master-2/wal/instance
I20260926 16:41:50.007905 11898 external_mini_cluster.cc:1428] Setting key 4ece9f28faf2bd4e4121ae19fba24ff6
I20260926 16:41:50.013131 2275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.190@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:41:50.017433 2200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } attempt: 3
I20260926 16:41:50.021344 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.005496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45944)
0926 16:41:50.005645 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:50.005663 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:50.005762 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:50.008296 (+ 2534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:50.008302 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:50.008554 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:50.008709 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.008719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.011072 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.011076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.011952 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.011965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.012998 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.013001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.013093 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.013759 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:50.013776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:50.015489 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:50.019034 (+ 3545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.019039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.019041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.019330 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.019747 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.019754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.019759 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.019859 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.020786 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:50.020791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:50.020904 (+ 113us) client_negotiation.cc:769] Sending connection context
0926 16:41:50.020995 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 16:41:50.021144 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:50.023353 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.008571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.11.158.188:33577)
0926 16:41:50.008874 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.008883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.009060 (+ 177us) server_negotiation.cc:407] Connection header received
0926 16:41:50.009182 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.009185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.009232 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.009359 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.010298 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.011077 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.012262 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.012837 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.013019 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.017115 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.017119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.017122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.017152 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.019134 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.019642 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.019645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.019647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.019726 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.020036 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.020039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.020040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.020655 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.020904 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.021901 (+ 997us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.022289 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":210,"server-negotiator.queue_time_us":235,"spinlock_wait_cycles":12928,"thread_start_us":127,"threads_started":1}
I20260926 16:41:50.023353 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.006341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.11.158.188:33577)
0926 16:41:50.006759 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.006784 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.006820 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:41:50.007554 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.007583 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.007969 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.008978 (+ 1009us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.009441 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.010926 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.012118 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.012838 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.013354 (+ 516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.015795 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.015807 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.015828 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.015872 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.018889 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.019469 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.019475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.019486 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.019638 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.019978 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.019984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.019987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.020655 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.021663 (+ 1008us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.021856 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.022238 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":273,"spinlock_wait_cycles":14464,"thread_start_us":144,"threads_started":1}
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.189@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:41:50.029600 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.016297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.11.158.190:41541)
0926 16:41:50.016487 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.016493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.016510 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:41:50.016622 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.016627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.016690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.016780 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.018113 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.018911 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.020274 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.020761 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.020844 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.023748 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.023754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.023759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.023793 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.026630 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.027872 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.027879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.027883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.027982 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.028341 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.028363 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.028365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.028668 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.028756 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.029271 (+ 515us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.029403 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260926 16:41:50.030138 2125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428: Bootstrap starting.
I20260926 16:41:50.030405 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.015382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.190:41541 (local address 127.0.0.1:46194)
0926 16:41:50.016129 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:50.016197 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:50.016316 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:50.016874 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:50.016881 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:50.017777 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:50.017968 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.017977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.019045 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.019048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.020133 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.020145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.021002 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.021005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.021107 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.021741 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:50.021756 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:50.023585 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:50.027352 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.027361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.027370 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.027748 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.028115 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.028118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.028120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.028220 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.028856 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:50.028865 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:50.029002 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 16:41:50.030014 (+ 1012us) client_negotiation.cc:240] Negotiation successful
0926 16:41:50.030213 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":128,"threads_started":1}
I20260926 16:41:50.030606 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.017968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45968)
0926 16:41:50.018229 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:50.018243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:50.018316 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:50.018976 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:50.018982 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:50.019177 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:50.019292 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.019299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.021014 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.021017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.021865 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.021878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.022564 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.022568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.022660 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.025351 (+ 2691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:50.025369 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:50.026889 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:50.028948 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.028951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.028953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.029227 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.029550 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.029553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.029555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.029644 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.030100 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:50.030104 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:50.030208 (+ 104us) client_negotiation.cc:769] Sending connection context
0926 16:41:50.030289 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:41:50.030442 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260926 16:41:50.031257 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.018462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.11.158.188:33577)
0926 16:41:50.018727 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.018736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.018748 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:41:50.018779 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.018781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.018830 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.019404 (+ 574us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.019490 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.020882 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.022021 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.022399 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.022886 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.027034 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.027038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.027040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.027065 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.028829 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.029354 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.029361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.029364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.029445 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.029767 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.029771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.029774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.029986 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.030782 (+ 796us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.030973 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.031099 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":207,"thread_start_us":107,"threads_started":1}
I20260926 16:41:50.031535 2275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.190@KRBTEST.COM
I20260926 16:41:50.035027 2200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: Bootstrap starting.
I20260926 16:41:50.035074 2125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:50.036092 2125 log.cc:824] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
I20260926 16:41:50.038065 2200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:50.038621 2125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428: No bootstrap required, opened a new log
I20260926 16:41:50.038945 2200 log.cc:824] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:50.041379 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.032135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38102 (local address 127.11.158.189:42679)
0926 16:41:50.032246 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.032251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.032301 (+ 50us) server_negotiation.cc:407] Connection header received
0926 16:41:50.032481 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.032484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.032529 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.032577 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.033140 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.033873 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.034949 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.035257 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.035334 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.037596 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.037600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.037603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.037640 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.039351 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.039859 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.039862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.039864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.039946 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.040249 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.040255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.040257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.040550 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.040650 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.041015 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.041138 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:41:50.041508 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.032079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.189:42679 (local address 127.0.0.1:38102)
0926 16:41:50.032244 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:50.032258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:50.032369 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:50.032711 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:50.032714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:50.032909 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:50.033014 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.033021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.034017 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.034020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.034799 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.034809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.035489 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.035492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.035576 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.035938 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:50.035952 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:50.037443 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:50.039480 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.039483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.039485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.039725 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.040050 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.040053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.040055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.040145 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.040697 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:50.040700 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:50.040813 (+ 113us) client_negotiation.cc:769] Sending connection context
0926 16:41:50.041205 (+ 392us) client_negotiation.cc:240] Negotiation successful
0926 16:41:50.041353 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260926 16:41:50.041970 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.027699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.11.158.190:41541)
0926 16:41:50.028532 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.028542 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.028555 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:41:50.028591 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.028595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.028647 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.028731 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.029588 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.030329 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.031486 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.031810 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.031916 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.035119 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.035124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.035127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.035155 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.037225 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.037745 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.037749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.037750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.037840 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.038399 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.038401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.038403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.040620 (+ 2217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.040831 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.041621 (+ 790us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.041761 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":99,"threads_started":1}
I20260926 16:41:50.041180 2200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: No bootstrap required, opened a new log
I20260926 16:41:50.042424 2275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:41:50.042860 2125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.043133 2125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7da15438994a4d3880a0a0b778527428, State: Initialized, Role: FOLLOWER
I20260926 16:41:50.043550 2125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.045109 2125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:50.046334 2200 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.046545 2200 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e87a42a6e349369c0d805c548e04b1, State: Initialized, Role: FOLLOWER
I20260926 16:41:50.046942 2200 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.047438 2285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.048666 2285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:50.049929 2200 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.158.189@KRBTEST.COM
Sep 26 16:41:50 dist-test-slave-p39x krb5kdc[2043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440909, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.158.188@KRBTEST.COM for kudu/127.11.158.188@KRBTEST.COM
I20260926 16:41:50.055140 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.043366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.11.158.189:42679)
0926 16:41:50.043462 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.043466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.043714 (+ 248us) server_negotiation.cc:407] Connection header received
0926 16:41:50.043891 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.043894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.043944 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.044013 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.044900 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.045609 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.046813 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.047151 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.047238 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.050649 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.050653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.050656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.050682 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.052486 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.053028 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.053032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.053034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.053126 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.053472 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.053477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.053479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.053706 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.053932 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.054774 (+ 842us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.054919 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:41:50.050674 2056 master_runner.cc:186] Error getting master registration for 127.11.158.189:42679: OK, instance_id { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" instance_seqno: 1790440909763824 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:41:50.057228 2056 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:41:50.058306 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.043205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.11.158.188:33577)
0926 16:41:50.047595 (+ 4390us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.047600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.047617 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:41:50.047655 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.047658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.047699 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.047788 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.048244 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.049001 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.051240 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.051953 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.052069 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.054814 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.054818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.054821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.054844 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.056592 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.057006 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.057010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.057012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.057091 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.057372 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.057375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.057378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.057606 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.057664 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.057967 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.058132 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":4304}
I20260926 16:41:50.058616 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.042905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.158.188:33577 (local address 127.0.0.1:45970)
0926 16:41:50.043279 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0926 16:41:50.043295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:41:50.043373 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:41:50.047826 (+ 4453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:41:50.047830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:41:50.048036 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:41:50.048134 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.048141 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.049997 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.050000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.050821 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:41:50.050834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.052226 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.052230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:41:50.052367 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.052824 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:41:50.052843 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:41:50.054450 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:41:50.056697 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.056699 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.056701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.056908 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.057176 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:41:50.057177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:41:50.057179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:41:50.057267 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:41:50.057730 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:41:50.057733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:41:50.057822 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 16:41:50.057927 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 16:41:50.058464 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293}
I20260926 16:41:50.058787 11898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 16:41:50.060146 2296 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 16:41:50.060122 2098 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7da15438994a4d3880a0a0b778527428"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46208
W20260926 16:41:50.060297 2296 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:50.060433 2098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 16:41:50.060412 2297 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.060536 2297 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:50.060544 2098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:50.061569 2098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.063010 2174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7da15438994a4d3880a0a0b778527428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e8e87a42a6e349369c0d805c548e04b1"
I20260926 16:41:50.063088 2098 leader_election.cc:288] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [CANDIDATE]: Term 1 election: Requested vote from peers e8e87a42a6e349369c0d805c548e04b1 (127.11.158.189:42679), 5a23d8bc174041d6b730691e4d13a4c5 (127.11.158.188:33577)
I20260926 16:41:50.063274 2174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:50.064778 2174 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7da15438994a4d3880a0a0b778527428 in term 1.
I20260926 16:41:50.065371 2072 leader_election.cc:302] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7da15438994a4d3880a0a0b778527428, e8e87a42a6e349369c0d805c548e04b1; no voters:
I20260926 16:41:50.065742 2285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:41:50.066190 2285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [term 1 LEADER]: Becoming Leader. State: Replica: 7da15438994a4d3880a0a0b778527428, State: Running, Role: LEADER
I20260926 16:41:50.066859 2285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.060067 2132 master_runner.cc:186] Error getting master registration for 127.11.158.188:33577: OK, instance_id { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" instance_seqno: 1790440910000913 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:41:50.067245 2132 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:41:50.069106 2275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: Bootstrap starting.
I20260926 16:41:50.070904 2316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7da15438994a4d3880a0a0b778527428. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.071058 2316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: This master's current role is: LEADER
W20260926 16:41:50.071465 2309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:50.071547 2309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:50.071743 2319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:41:50.072100 2275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: Neither blocks nor log segments found. Creating new log.
I20260926 16:41:50.073089 2275 log.cc:824] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: Log is configured to *not* fsync() on all Append() calls
I20260926 16:41:50.073858 2319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:41:50.077919 2275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: No bootstrap required, opened a new log
I20260926 16:41:50.083568 2275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.083901 2275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a23d8bc174041d6b730691e4d13a4c5, State: Initialized, Role: FOLLOWER
I20260926 16:41:50.084280 2275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } }
I20260926 16:41:50.085956 2275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:41:50.086030 2249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7da15438994a4d3880a0a0b778527428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5a23d8bc174041d6b730691e4d13a4c5"
I20260926 16:41:50.086256 2249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:41:50.087836 2249 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7da15438994a4d3880a0a0b778527428 in term 1.
I20260926 16:41:50.101975 2249 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [term 1 FOLLOWER]: Refusing update from remote peer 7da15438994a4d3880a0a0b778527428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:50.102715 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.068576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.11.158.189:42679)
0926 16:41:50.068702 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.068707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.073667 (+ 4960us) server_negotiation.cc:407] Connection header received
0926 16:41:50.073857 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.073861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.073913 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.073968 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.077140 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.077931 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.079133 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.079480 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.079592 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.089665 (+ 10073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.089671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.089675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.089715 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.093012 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.093549 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.093554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.093556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.093656 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.101302 (+ 7646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.101306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.101308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.101579 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.101720 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.102385 (+ 665us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.102546 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:41:50.102824 2316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:41:50.105295 2334 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:41:50.105383 2334 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:41:50.105589 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.068590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46216 (local address 127.11.158.190:41541)
0926 16:41:50.068711 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.068715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.071968 (+ 3253us) server_negotiation.cc:407] Connection header received
0926 16:41:50.072218 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.072222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.072296 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.072398 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.075017 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.075853 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.079158 (+ 3305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.079546 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.079630 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.081741 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.081747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.081750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.081803 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.083711 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.093787 (+ 10076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.093791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.093793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.093893 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.100716 (+ 6823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.100722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.100723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.101058 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.101195 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.105266 (+ 4071us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.105432 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:41:50.107550 2336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.107690 2336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:50.107788 2336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7da15438994a4d3880a0a0b778527428. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.107862 2336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:50.108846 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:50.073830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.11.158.188:33577)
0926 16:41:50.073972 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.073976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.075016 (+ 1040us) server_negotiation.cc:407] Connection header received
0926 16:41:50.075201 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.075204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.075251 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.075305 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.084453 (+ 9148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.085188 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.090488 (+ 5300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:50.090927 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:50.091061 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:50.094147 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:41:50.094152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:41:50.094155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:41:50.094210 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:41:50.102497 (+ 8287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.106127 (+ 3630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.106131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.106132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.106238 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:41:50.107416 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:41:50.107420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:41:50.107422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:41:50.107684 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:41:50.107830 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:50.108554 (+ 724us) server_negotiation.cc:299] Negotiation successful
0926 16:41:50.108684 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:41:50.109659 2174 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [term 1 FOLLOWER]: Refusing update from remote peer 7da15438994a4d3880a0a0b778527428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:41:50.110293 2285 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:41:50.115854 2297 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7da15438994a4d3880a0a0b778527428. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.116286 2297 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:50.118772 2316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.118911 2316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:50.119791 2336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.119913 2336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:50.120786 2337 mvcc.cc:205] Tried to move back new op lower bound from 7333645967749140480 to 7333645967641706496. Current Snapshot: MvccSnapshot[applied={T|T < 7333645967749140480}]
I20260926 16:41:50.120853 2319 catalog_manager.cc:1347] Generated new cluster ID: abb8990a279a493b964de169a1c255ad
I20260926 16:41:50.120915 2319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:41:50.124425 2322 mvcc.cc:205] Tried to move back new op lower bound from 7333645967749140480 to 7333645967641706496. Current Snapshot: MvccSnapshot[applied={T|T < 7333645967749140480 or (T in {7333645967749140480})}]
I20260926 16:41:50.128721 2285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.128868 2285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428 [sys.catalog]: This master's current role is: LEADER
I20260926 16:41:50.130082 2297 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7da15438994a4d3880a0a0b778527428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7da15438994a4d3880a0a0b778527428" member_type: VOTER last_known_addr { host: "127.11.158.190" port: 41541 } } peers { permanent_uuid: "e8e87a42a6e349369c0d805c548e04b1" member_type: VOTER last_known_addr { host: "127.11.158.189" port: 42679 } } peers { permanent_uuid: "5a23d8bc174041d6b730691e4d13a4c5" member_type: VOTER last_known_addr { host: "127.11.158.188" port: 33577 } } }
I20260926 16:41:50.130218 2297 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:41:50.131135 2339 mvcc.cc:205] Tried to move back new op lower bound from 7333645967749140480 to 7333645967641706496. Current Snapshot: MvccSnapshot[applied={T|T < 7333645967749140480}]
I20260926 16:41:50.159271 2319 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:41:50.160761 2319 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:41:50.178179 2319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7da15438994a4d3880a0a0b778527428: Generated new TSK 0
I20260926 16:41:50.179250 2319 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:41:50.210390 2281 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.206560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.11.158.188:33577)
0926 16:41:50.206691 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.206696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.208932 (+ 2236us) server_negotiation.cc:407] Connection header received
0926 16:41:50.209420 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.209424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.209479 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.209547 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.210237 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45978: BlockingRecv error: recv got EOF from 127.0.0.1:45978 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 16:41:50.212085 2282 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.207454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46226 (local address 127.11.158.190:41541)
0926 16:41:50.207569 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.207574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.211092 (+ 3518us) server_negotiation.cc:407] Connection header received
0926 16:41:50.211270 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.211273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.211325 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.211377 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.211938 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46226: BlockingRecv error: recv got EOF from 127.0.0.1:46226 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 16:41:50.213846 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.208478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38122 (local address 127.11.158.189:42679)
0926 16:41:50.208663 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.208668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.212786 (+ 4118us) server_negotiation.cc:407] Connection header received
0926 16:41:50.213087 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.213091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.213143 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.213194 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.213727 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38122: BlockingRecv error: recv got EOF from 127.0.0.1:38122 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260926 16:41:50.218809 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.216545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.11.158.189:42679)
0926 16:41:50.216657 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.216661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.217762 (+ 1101us) server_negotiation.cc:407] Connection header received
0926 16:41:50.218017 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.218022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.218106 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.218159 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.218702 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38128: BlockingRecv error: recv got EOF from 127.0.0.1:38128 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 16:41:50.224069 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.222565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.11.158.189:42679)
0926 16:41:50.222672 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.222676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.223157 (+ 481us) server_negotiation.cc:407] Connection header received
0926 16:41:50.223342 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.223345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.223410 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.223463 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.223954 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38140: BlockingRecv error: recv got EOF from 127.0.0.1:38140 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 16:41:50.230075 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.228597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.11.158.189:42679)
0926 16:41:50.228701 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.228705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.229170 (+ 465us) server_negotiation.cc:407] Connection header received
0926 16:41:50.229363 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.229367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.229418 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.229470 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.229962 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38146: BlockingRecv error: recv got EOF from 127.0.0.1:38146 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 16:41:50.238224 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.236628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.11.158.189:42679)
0926 16:41:50.236774 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.236779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.237280 (+ 501us) server_negotiation.cc:407] Connection header received
0926 16:41:50.237488 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.237492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.237544 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.237594 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.238107 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38148: BlockingRecv error: recv got EOF from 127.0.0.1:38148 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 16:41:50.251739 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.249904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.11.158.189:42679)
0926 16:41:50.250032 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.250037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.250603 (+ 566us) server_negotiation.cc:407] Connection header received
0926 16:41:50.250851 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.250855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.250923 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.250984 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.251612 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38154: BlockingRecv error: recv got EOF from 127.0.0.1:38154 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 16:41:50.272955 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.270942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.11.158.189:42679)
0926 16:41:50.271081 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.271086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.271724 (+ 638us) server_negotiation.cc:407] Connection header received
0926 16:41:50.271972 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.271977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.272067 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.272126 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.272819 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38156: BlockingRecv error: recv got EOF from 127.0.0.1:38156 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 16:41:50.309319 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.307717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38172 (local address 127.11.158.189:42679)
0926 16:41:50.307848 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.307853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.308131 (+ 278us) server_negotiation.cc:407] Connection header received
0926 16:41:50.308561 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.308572 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.308625 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.308718 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.309189 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38172: BlockingRecv error: recv got EOF from 127.0.0.1:38172 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 16:41:50.377199 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.376014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.11.158.189:42679)
0926 16:41:50.376199 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.376203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.376224 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:41:50.376440 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.376444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.376496 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.376563 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.377065 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38176: BlockingRecv error: recv got EOF from 127.0.0.1:38176 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260926 16:41:50.508605 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.507430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.11.158.189:42679)
0926 16:41:50.507614 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.507618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.507715 (+ 97us) server_negotiation.cc:407] Connection header received
0926 16:41:50.507857 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.507859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.507909 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.507986 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.508465 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38188: BlockingRecv error: recv got EOF from 127.0.0.1:38188 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260926 16:41:50.768028 2204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:41:50.766672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.11.158.189:42679)
0926 16:41:50.766832 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:50.766836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:50.767024 (+ 188us) server_negotiation.cc:407] Connection header received
0926 16:41:50.767148 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:50.767151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:50.767203 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:50.767288 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:41:50.767912 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38192: BlockingRecv error: recv got EOF from 127.0.0.1:38192 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 16:41:51.073014 2309 catalog_manager.cc:1259] Loaded cluster ID: abb8990a279a493b964de169a1c255ad
I20260926 16:41:51.073104 2309 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: loading cluster ID for follower catalog manager: success
I20260926 16:41:51.074800 2309 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: acquiring CA information for follower catalog manager: success
I20260926 16:41:51.075970 2309 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8e87a42a6e349369c0d805c548e04b1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:41:51.106756 2334 catalog_manager.cc:1259] Loaded cluster ID: abb8990a279a493b964de169a1c255ad
I20260926 16:41:51.106849 2334 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: loading cluster ID for follower catalog manager: success
I20260926 16:41:51.108630 2334 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: acquiring CA information for follower catalog manager: success
I20260926 16:41:51.109972 2334 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5a23d8bc174041d6b730691e4d13a4c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:41:51.287055 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:51.282636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38198 (local address 127.11.158.189:42679)
0926 16:41:51.282914 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:51.282921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:51.283022 (+ 101us) server_negotiation.cc:407] Connection header received
0926 16:41:51.283209 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:51.283215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:51.283369 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:51.283495 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:51.283794 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.284363 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.285275 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.285540 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.285680 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:51.286046 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:51.286626 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:51.286766 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:51.286810 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 16:41:51.286879 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260926 16:41:51.295075 2414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:51.290517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.11.158.188:33577)
0926 16:41:51.290746 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:51.290750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:51.290765 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:41:51.291004 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:51.291008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:51.291138 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:51.291247 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:51.291544 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.292192 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.293277 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.293551 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.293656 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:51.294033 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:51.294538 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:51.294648 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:51.294793 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 16:41:51.294859 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260926 16:41:51.303185 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:51.298515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46238 (local address 127.11.158.190:41541)
0926 16:41:51.298782 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:51.298789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:51.298900 (+ 111us) server_negotiation.cc:407] Connection header received
0926 16:41:51.299093 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:51.299098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:51.299289 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:51.299425 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:51.299686 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.300568 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.301536 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.301784 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.301906 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:51.302373 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:51.302750 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:51.302861 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:51.302904 (+ 43us) server_negotiation.cc:299] Negotiation successful
0926 16:41:51.302983 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":76,"threads_started":1}
I20260926 16:41:51.303304 2414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:51.299018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.11.158.188:33577)
0926 16:41:51.299160 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:51.299164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:51.299333 (+ 169us) server_negotiation.cc:407] Connection header received
0926 16:41:51.299527 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:51.299531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:51.299581 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:51.299628 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:51.299982 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.300556 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.302072 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.302424 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.302526 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:51.302785 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:51.302878 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:51.302955 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:51.303100 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 16:41:51.303168 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:41:51.303426 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:41:51.299153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38206 (local address 127.11.158.189:42679)
0926 16:41:51.299270 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:41:51.299277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:41:51.299462 (+ 185us) server_negotiation.cc:407] Connection header received
0926 16:41:51.299625 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:41:51.299628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:41:51.299688 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:41:51.299769 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:41:51.300033 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.300865 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.302265 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:41:51.302547 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:41:51.302643 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:41:51.302918 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:41:51.303002 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:41:51.303049 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 16:41:51.303213 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 16:41:51.303278 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 16:41:51.307367 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 2056
I20260926 16:41:51.314431 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 2132
I20260926 16:41:51.320650 11898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskupRbDL/build/debug/bin/kudu with pid 2206
2026-09-26T16:41:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3124 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6294 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288377 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST